AFRL-RI-RS-TR-2007-271 
Final  Technical  Report 
December  2007 


LEARNING  AND  SELF-REPAIRING  SYSTEMS 


University  of  Florida 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


STINFO  COPY 


AIR  FORCE  RESEARCH  LABORATORY 
INFORMATION  DIRECTORATE 
ROME  RESEARCH  SITE 
ROME,  NEW  YORK 


NOTICE  AND  SIGNATURE  PAGE 


Using  Government  drawings,  specifications,  or  other  data  included  in  this  document  for 
any  purpose  other  than  Government  procurement  does  not  in  any  way  obligate  the  U.S. 
Government.  The  fact  that  the  Government  formulated  or  supplied  the  drawings, 
specifications,  or  other  data  does  not  license  the  holder  or  any  other  person  or 
corporation;  or  convey  any  rights  or  permission  to  manufacture,  use,  or  sell  any  patented 
invention  that  may  relate  to  them. 

This  report  was  cleared  for  public  release  by  the  Air  Force  Research  Laboratory  Public 
Affairs  Office  and  is  available  to  the  general  public,  including  foreign  nationals.  Copies 
may  be  obtained  from  the  Defense  Technical  Information  Center  (DTIC) 
(http://www.dtic.mil). 


AFRL-RI-RS-TR-2007-27 1  HAS  BEEN  REVIEWED  AND  IS  APPROVED  FOR 
PUBLICATION  IN  ACCORDANCE  WITH  ASSIGNED  DISTRIBUTION 
STATEMENT. 


FOR  THE  DIRECTOR: 

/s/ 


/s/ 


CHRISTOPHER  FLYNN  JAMES  A.  COLLINS,  Deputy  Chief 

Work  Unit  Manager  Advanced  Computing  Division 

Information  Directorate 


This  report  is  published  in  the  interest  of  scientific  and  technical  infonnation  exchange,  and  its 
publication  does  not  constitute  the  Government’s  approval  or  disapproval  of  its  ideas  or  findings. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 

OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  data  sources, 

gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection 

of  information,  including  suggestions  for  reducing  this  burden  to  Washington  Headquarters  Service,  Directorate  for  Information  Operations  and  Reports, 

1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget, 

Paperwork  Reduction  Project  (0704-0188)  Washington,  DC  20503. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


5b.  GRANT  NUMBER 

FA8750-06-1-0175 

5c.  PROGRAM  ELEMENT  NUMBER 


5e.  TASK  NUMBER 

10 

5f.  WORK  UNIT  NUMBER 

14 


12.  DISTRIBUTION  AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED.  PA #  WPAFB  07-0686 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 

The  research  covered  by  this  grant  concentrated  on  the  development  of  computing  algorithms  for  learning  and  self-repairing  systems. 
During  the  report  period,  several  existing  and  new  methodologies  were  critically  examined.  The  research  resulted  in  the 
development  of  the  concepts  of  learning  blocks  and  association  degree,  which  facilitate  the  development  of  learning  and  self¬ 
repairing  systems.  Methodologies  based  on  these  concepts  allow  a  system  to  extract  critical  information  from  its  past  operation  to 
automatically  generate  remedies  for  future  malfunctions. 


15.  SUBJECT  TERMS 

Learning,  Self-repairing  systems,  genetic  algorithms 


17.  LIMITATION  OF  18.  NUMBER 
ABSTRACT  OF  PAGES 

UL  52 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


19a.  NAME  OF  RESPONSIBLE  PERSON 

Christopher  Flynn 

19b.  TELEPHONE  NUMBER  ( Include  area  code) 

N/A 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT  I  b.  ABSTRACT  I  c.  THIS  PAGE 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

University  of  Florida 
319  Weil  HL 

Gainesville  FL  32611-5500 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

AFRL/RITB 
525  Brooks  Rd 
Rome  NY  13441-4505 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 


11.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 

AFRL-RI-RS-TR-2007-271 


5d.  PROJECT  NUMBER 

NBGQ 


3.  DATES  COVERED  (From  -  To) 
Jul  06  -  Jun  07 

5a.  CONTRACT  NUMBER 


1.  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE 

DEC  2007  Final 

4.  TITLE  AND  SUBTITLE 

LEARNING  AND  SELF-REPAIRING  SYSTEM 

6.  AUTHOR(S) 

Jacob  Hammer 


TABLE  OF  CONTENTS 


1.  SUMMARY  1 

2.  THE  DEMONSTRATION  SYSTEM  2 

3.  THE  OPTIMIZATION  CRITERION  3 

4.  IMPLEMENTATION  OF  THE  GENETIC  ALGORITHM  4 

5.  LEARNING  BLOCKS  6 

6.  THE  LEARNING  PROCESS  AND  THE  ASSOCIATION  DEGREE  9 

7.  ADAPTIVE  CONTROL  OF  LEARNING  SYSTEMS  11 

8.  PUBLICATIONS  14 

9.  CONCLUSIONS  15 

10.  APPENDIX:  PUBLICATIONS  16 


1 


1.  SUMMARY 


The  research  that  was  conducted  on  this  project  fulfilled  all  the  objectives  set 
forward  at  the  start  of  the  project.  We  have  developed  a  simulation  system  that 
demonstrates  the  advantages  of  our  learning  and  self-repair  methodology  and 
we  have  initiated  a  theoretical  study  into  the  foundations  of  a  general  design 
methodology  based  on  learning  and  self-repair.  The  theoretical  foundation  is  built 
around  a  framework  of  clockless  logic  sequential  machines,  allowing  the  theory 
to  be  applied  to  extensively  parallel  computing  systems. 

a)  Development  and  construction  of  a  simulation  system:  We  have  a 
simulation  system  that  demonstrates  the  advantages  of  learning  and  self  repair 
over  the  existing  technology  of  genetic  algorithms.  The  system  demonstrates  in 
the  clear  visual  way  the  disadvantages  of  genetic  algorithms:  the  emergence  of 
non-viable  solutions  during  the  process  of  chromosome  breeding  and  mutation; 
the  long  time  genetic  algorithms  take  to  approach  optimal  performance  in  high 
complexity  systems;  and  the  lack  of  learning,  which  requires  the  system  to  start 
from  scratch  after  every  significant  failure.  These  disadvantages  bring  to  light  the 
superiority  of  the  learning  block  approach  we  undertook,  and  point  to  the 
importance  of  continuing  the  development  of  learning  methodologies.  Here  is  a 
brief  review  of  the  setup  of  our  genetic  algorithm  test  framework. 

The  simulation  test  bed  we  used  simulates  the  control  of  the  traffic  in  a 
computer  communication  network.  The  simulation  system  was  developed 
specifically  for  the  simulation  of  self-repair  algorithms  for  large  scale  distributed 
computing  clusters,  the  self-repair  of  computer  communication  networks,  and 
self-repair  of  difficulties  in  the  cooperative  operation  of  multiple  combat  units.  The 
simulation  system  consists  of  nodes  and  links,  where  the  nodes  control  the  traffic 
flow  through  the  links.  In  the  current  format,  each  node  is  connected  to  four 
branches  that  emerge  from  the  node,  where  each  branch  allows  bi-directional 
traffic.  Four  control  handles  are  provided  at  each  node,  allowing  to  control  the 
flow  rate  of  each  of  the  four  exit  gates  at  each  node.  The  traffic  flow  rate  through 
each  gate  can  be  controlled  independently.  In  addition,  each  node  generates 
traffic  that  enters  the  network  at  the  node,  simulating  data,  instructions,  or 
computing  results  that  originate  at  the  node. 

In  the  case  of  distributed  computing  clusters,  each  node  represents  a 
computing  center  of  the  cluster,  while  the  links  represent  the  flow  of  data, 
instructions,  and  results  from  one  cluster  to  another.  In  the  case  of  a  computer 
communication  network,  the  nodes  represent  routers  and  the  links  represent 
conduits  of  the  network.  In  the  case  of  cooperating  combat  units,  the  nodes 
represent  individual  units  and  the  links  represent  the  flow  of  assigned  tasks 
among  the  units. 
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2.  THE  DEMONSTRATION  SYSTEM 


The  demonstration  program  that  was  used  includes  30  nodes  and  more  than 
50  links.  Thus,  it  represents  a  distributed  computing  cluster  with  30  computing 
centers.  Each  node  is  capable  of  emitting  traffic  into  the  network,  representing 
the  instructions  and  data  generated  at  the  computing  cluster  or  computing 
network  node  it  represents.  In  addition,  each  gate  at  each  node  has  a  queue, 
indicating  elements  (instructions  or  data  packets)  waiting  to  leave  the  node  in  the 
corresponding  direction.  All  the  algorithms  that  are  implemented  in  the  system 
are  scalable,  so  larger  computing  clusters  can  be  managed  with  the  same 
algorithms. 

The  status  of  each  node  in  the  simulation  is  represented  by  a  four  digit  string 
of  digits  (a  quadruplet),  where  each  digit  represents  the  status  of  one  of  the  four 
gates  at  the  node:  the  first  digit  represents  the  west  gate,  the  second  digit 
represents  the  north  gate,  the  third  digit  represents  the  east  gate,  and  the  fourth 
digit  represents  the  south  gate.  Each  digit  is  an  integer  between  0  and  5,  where 
a  digit  c  represents  a  flow  rate  of  0.2c  per  second,  so  that  the  flow  rate  varies 
between  zero  (closed  gate)  and  1  (fully  open  gate).  For  example,  the  string 
(0130)  indicates  that  the  west  gate  is  closed,  the  north  gate  allows  a  flow  of 
0.2/sec,  the  east  gate  allows  a  flow  of  0.6/sec,  and  the  south  gate  is  closed. 

In  addition,  malfunctions  at  a  node  are  represented  by  the  character  N  which 
indicates  that  the  corresponding  gate  of  the  node  is  permanently  closed.  Thus, 
the  string  (00N0)  indicates  that  the  east  gate  of  the  node  is  permanently  closed; 
the  west,  north,  and  south  gates  are  also  closed  at  this  time,  but  the  algorithm 
can  open  them  in  its  attempt  to  create  a  more  efficient  flow  through  the 
simulation.  In  this  way,  the  status  of  the  entire  system  is  then  represented  by  a 
string  of  30  quadruplets;  for  example,  the  string 

(0240)(N501 ) . (003N). 

Each  such  string  represents  a  possible  control  option  for  the  traffic  flow  in  our 
network.  It  is  often  referred  to  as  a  chromosome. 

To  implement  a  genetic  algorithm,  we  start  from  an  initial  population  So  of  4 
chromosomes  Ci,  C2,  C3,  and  C4.  Except  for  the  locations  of  the  characters  N, 
which  represent  defective  gates,  the  characters  in  each  one  of  the  four  initial 
chromosomes  are  selected  randomly. 
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3.  THE  OPTIMIZATION  CRITERION: 


When  the  simulation  reaches  a  steady  state,  the  criterion  is  taken  as 
J  :=  number  of  queues  that  contain  more  than  5  elements  in  steady  state. 

Note  that  the  values  of  J  are  integers. 

Step  G1 :  The  system  is  run  to  steady  state  with  each  one  of  the  four  initial 
chromosomes,  and  the  value  of  the  criterion  J  is  calculated  in  each  case.  This 
yields  the  four  criterion  values  J-i,  J2,  J3,  and  J4.  (If  one  of  the  chromosomes 
does  not  yield  as  steady  state,  then  it  is  discarded  and  replaced  by  a  randomly 
selected  new  chromosome.) 

Step  G2:  The  next  step  of  the  genetic  algorithm  is  the  reproduction  process.  We 
produce  multiple  copies  of  our  strings,  generating  more  copies  of  the  strings  that 
yield  a  better  criterion  value.  Since  we  are  seeking  a  minimum  in  this  case,  the 
number  of  copies  of  string  Cj  is  selected  in  proportion  to  1/Jj.  For  a  real  number 
a,  let  [a]1  be  the  integer  obtained  by  truncating  a.  We  then  create 

[100/Ji]1  copies  of  the  chromosome  Ci; 

[100/JT  copies  of  the  chromosome  C2; 

[lOO/Jsf  copies  of  the  chromosome  C3; 

[100/J4]1  copies  of  the  chromosome  C4; 

This  creates  a  set  Si  of  chromosomes. 

Step  G3:  The  next  step  is  the  crossover  step:  We  select  randomly  four  pairs  of 
chromosomes  from  the  set  Si.  Let's  denote  the  resulting  pairs  by  (Cn,C21), 
(Ci2,C22),  (Ci3,C23),  (C14,C24).  For  each  one  of  these  pairs,  we  select  a  random 
integer  1  <  Nk  ^  30,  k  =  1 ,  2,  3,  4.  We  then  create  from  each  pair  a  new 
chromosome,  using  the  following  process:  for  the  pair  (Cik,C2k),  we  use  the  first 
Nk  quadruplets  of  C1k  and  the  last  (30  -  Nk)  quadruplets  of  C2k.  Denote  by  Ck 
the  resulting  chromosome,  k  =  1 ,  2,  3,  4.  This  yields  a  new  set  of  four 
chromosomes. 

Step  G4:  The  next  step  is  the  mutation  step:  create  a  new  string  S  :=  C}C2C3C4 
of  480  bits  by  concatenating  the  four  chromosomes.  Pick  a  random  integer  r  in 
the  range  [1,  10000],  If  1<r<480  and  if  position  r  in  S  is  not  N,  then  switch 
the  value  of  character  r  of  S  according  to  the  following  table: 


Original 

Value 

Switch 

to 

0 

1 

3 


1 

2 

2 

3 

3 

4 

4 

5 

5 

0 

N 

N 

1111 

Finally,  divide  S  back  into  4  chromosomes  C-,,  C2,  C3,  C4.  On  average,  this 

process  results  in  the  change  of  one  gate  value  every  twenty  runs.  Repeat  from 
Step  G1 . 

This  process  continues  until  a  maximal  value  of  J  is  obtained. 


4.  IMPLEMENTATION  OF  THE  GENETIC  ALGORITHM 


The  genetic  algorithm  framework  described  above  was  implemented  and  run 
in  a  real  time  simulation,  with  the  objective  to  optimize  system  performance  after 
various  failures.  The  simulations  pointed  to  fundamental  deficiencies  of  the 
genetic  algorithm  approach  to  the  problem  of  controlling  a  large  digital  network 
that  is  subject  to  defects  and  failures. 

1 )  One  of  the  fundamental  weaknesses  of  genetic  algorithms  in  our  simulations 
turned  out  to  be  the  process  of  finding  the  initial  four  chromosomes.  Recall  that 
the  initial  four  chromosomes  Ci,  C2,  C3,  and  C4  are  found  through  a  random 
process.  However,  in  our  simulations,  the  vast  majority  of  randomly  constructed 
chromosomes  did  not  lead  to  a  steady  state  of  the  traffic  in  our  simulated 
computer  network.  Instead,  the  vast  majority  of  randomly  selected  chromosomes 
lead  to  conditions  of  network  congestion  and  continually  increasing  queues  at  the 
network  gates.  Such  chromosomes  are,  of  course,  not  suitable  for  the  operation 
of  our  traffic  network,  and  they  cannot  be  used  as  a  basis  of  the  process  of 
breeding  new  chromosomes.  They  have  been  discarded,  and  new  chromosomes 
have  been  selected  and  tested.  In  most  cases,  the  simulation  had  to  run  through 
a  rather  large  number  of  randomly  selected  chromosomes,  before  it  was  able  to 
find  four  chromosomes  that  control  the  network  to  a  steady  state  and  do  not 
induce  congestion.  Furthermore,  the  determination  of  steady  state  is  a  rather 
lengthy  process  -  the  system  has  to  run  for  a  period  of  time  before  it  can  be 
determined  that  steady  state  has  been  reached.  As  this  process  has  to  be 
repeated  after  every  system  failure,  it  is  quite  obvious  that  genetic  algorithms  do 
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not  facilitate  fast  recovery  of  the  computing  network  after  failure,  and  hence  are 
not  suitable  for  the  application  at  hand. 

2)  The  process  of  breeding  the  chromosomes  created  similar  difficulties.  For  the 
size  of  the  network  we  are  currently  using  (30  nodes  and  50  links)  and  under 
high  traffic  conditions,  it  happened  rather  often  that  the  process  of  randomly 
breeding  two  chromosomes  did  not  yield  a  viable  chromosome.  In  other  words, 
very  often  chromosomes  that  resulted  from  the  breeding  process  lead  to  traffic 
congestion  in  the  network,  and  did  not  lead  the  network  toward  steady  state 
traffic  conditions.  Thus,  it  seems  that,  for  high  complexity  systems,  the  process  of 
chromosome  breeding  can  become  dysfunctional.  In  such  case,  we  had  to  return 
to  the  random  selection  process  to  create  a  new  chromosome.  The  latter 
process,  as  indicated  earlier,  required  a  lengthy  and  time  consuming  process,  as 
part  of  the  attempt  to  find  a  chromosome  that  controls  the  traffic  in  the  network 
without  creating  conditions  of  congestions  and  allows  the  network  to  reach  a 
steady  state. 

3)  The  random  mutation  process  that  forms  an  integral  part  of  the  genetic 
algorithm  process  encountered  similar  difficulties.  Sometimes,  under  heavy 
network  traffic  conditions,  genetic  mutation  that  was  applied  to  a  good 
chromosome  resulted  in  a  chromosome  that  caused  conditions  of  traffic 
congestion  within  our  computing  network  and  had  to  be  discarded.  As  before, 
such  an  occurrence  forced  us  to  default  back  to  the  random  selection  process  in 
order  to  obtain  the  missing  chromosome.  As  indicated  earlier,  the  latter  process 
was  fraught  with  uncertainty,  often  yielding  dysfunctional  chromosomes. 

4)  Recall  that  one  of  the  main  objectives  of  this  research  project  is  to  develop  a 
control  mechanism  that  allows  the  system  to  recover  quickly  after  a  significant 
malfunction.  A  significant  malfunction  in  our  case  refers  to  a  situation  where  15% 
to  30%  of  the  network's  nodes  become  dysfunctional.  To  simulate  such  a 
situation,  we  randomly  assign  the  character  N  indicating  a  permanently  blocked 
node  gate  to  a  random  fraction  of  the  nodes,  varying  between  15%  and  30%.  In 
our  case,  the  network  has  a  total  of  30  nodes;  as  each  node  has  4  gates,  we 
have  a  total  of  120  gates.  To  simulate  failure,  we  block  a  random  selection  of  18 
to  36  nodes  out  of  the  total  of  1 20  gates,  where  the  number  of  blocked  nodes  is 
selected  randomly.  The  objective,  of  course,  is  to  evaluate  the  capability  of  the 
genetic  algorithm  to  return  the  system  to  proper  function. 

In  this  case  as  well,  genetic  algorithms  do  not  seem  to  be  an  ideal  choice  for 
the  task.  After  failure  simulation  was  applied  to  our  network,  the  existing 
chromosomes  often  did  not  perform  properly,  leading  the  network  to  conditions  of 
traffic  congestion.  As  a  result,  the  existing  chromosomes  could  not  be  used,  and 
the  entire  chromosome  generation  process  had  to  be  re-run.  As  indicated  earlier, 
this  is  a  rather  time  consuming  process. 

It  seems  that  genetic  algorithms  do  not  learn  and  do  not  accumulate  the 
experience  necessary  to  operate  a  system  in  the  presence  of  failure.  A  genetic 
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algorithm  does  not  seem  to  gather  intimate  knowledge  about  the  system 
structure,  knowledge  that  can  be  used  to  help  the  system  recover  quickly  from 
significant  malfunction.  Genetic  algorithms  do  not  seem  to  learn  the  system's 
structure  and  do  not  seem  capable  of  helping  implement  a  quick  recovery  after 
major  failure.  On  the  contrary,  attempts  to  control  the  system  after  significant 
failure  by  using  the  results  of  an  earlier  run  of  a  genetic  algorithms  often  seem  to 
worsen  the  system's  condition  after  failure  rather  than  leading  the  system  back  to 
steady  operation. 

To  summarize,  it  seems  that  genetic  algorithms  are  not  suitable  for  running 
systems  that  may  vary  significantly  over  the  course  of  time,  as  is  the  case  with 
systems  prone  to  failure.  Especially,  for  systems  that  are  subject  to  significant 
failure  and  malfunction,  genetic  algorithms  do  not  provide  a  general  path  to 
recovery.  In  most  cases,  after  each  malfunction  or  failure,  the  entire  genetic 
algorithm  process  needs  to  re-run  anew  in  order  to  return  the  computing  network 
to  proper  function.  Along  this  process,  the  process  of  breeding  new  genetic 
algorithms  often  destabilizes  the  system  and  leads  to  network  congestion, 
resulting  in  an  extended  period  of  time  during  which  the  system  is  not 
operational.  Thus,  a  new  approach  must  be  developed  to  allow  complex  systems 
to  recover  automatically  from  significant  failure. 


5.  LEARNING  BLOCKS 


The  learning  block  methodology  that  is  being  developed  as  part  of  this 
research  project  offers  substantial  advantages  when  it  comes  to  overcoming  the 
effects  of  failure  on  a  complex  system.  With  the  learning  block  methodology, 
knowledge  about  the  structure  of  the  system  is  collected  and  accumulated,  and 
this  knowledge  is  automatically  used  by  the  system  to  overcome  the  effects  of 
failures  and  defects.  The  learning  system  develops  a  database  that  relates 
various  aspects  of  performance  deterioration  to  the  corrective  actions  taken  in 
the  past  against  such  failures  and  malfunctions.  This  database  allows  the  system 
to  apply  the  correct  remedy  after  a  malfunction  or  failure  with  only  a  rare  process 
of  trial  and  error,  and  to  return  the  network  quickly  to  proper  operation.  This 
database  is  stored  within  subroutines  of  the  main  learning  algorithm. 

Before  discussing  the  learning  process  to  be  used  in  our  learning  block 
simulation  platform,  we  briefly  review  a  few  specifics  about  the  simulation 
platform  itself.  Recall  that  our  simulation  platform  consists  of  a  computer 
communication  network  with  30  nodes  and  about  50  links.  Each  node  has  four 
gates,  which  control  the  flow  of  data  and  instructions  into  the  four  links  that 
connect  to  the  node.  The  state  of  the  computing  network  is  therefore 
characterized  by  a  string  of  120  characters,  which  each  character  describes  the 
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status  of  one  of  the  network's  gates. 

In  this  context,  a  learning  block  represents  a  list  of  gates  with  their 
corresponding  transmission  rates.  In  other  words,  each  learning  block  assigns 
flow  rates  to  a  certain  subset  of  gates  in  the  network.  To  represent  our  learning 
blocks,  we  number  the  gates  of  our  computing  network  and  designate  them  by 
the  characters  gi,  g2,  ...,  gi2o-  Recalling  that  the  flow  rate  through  a  gate  is 
indicated  by  one  of  the  integers  1,  2,  3,  4,  5  or  by  the  character  N  which 
indicates  a  disabled  gate,  we  combine  these  symbols  so  that,  for  example,  the 
designation  g7(3)  indicates  that  gate  7  is  open  to  flow  rate  3,  or  g2(N) 
indicates  that  gate  2  is  disabled.  In  these  terms,  a  learning  block  is  a  string 
9a(ja)  gb(jb)...g/(jf)  -  it  simply  indicates  the  flow  rates  of  the  listed  gates,  and  has 

no  impact  on  other  gates.  Thus,  for  example,  the  string  g3(4)g6(2)g25(5) 
indicates  that  the  flow  through  gate  3  is  set  to  4;  the  flow  through  gate  2  is  set 
to  6;  and  the  flow  through  gate  25  is  set  to  5. 

Comparing  the  learning  blocks  to  chromosomes,  recall  that  the  chromosome 
in  this  case  is  a  string  of  120  characters,  where  each  character  describes  the 
flow  rate  through  one  of  the  gates  of  the  network.  When  a  chromosome  is 
activated,  it  impacts  the  flow  through  all  the  gates  in  the  computing  network.  The 
learning  block,  on  the  other  hand,  affects  only  the  gates  that  appear  in  its  string, 
and  has  no  impact  on  the  other  gates  of  the  network. 

Once  a  learning  block  is  applied,  we  wait  for  the  network  to  achieve  steady 
state,  and  the  impact  of  the  learning  block  on  the  gate  queues  in  the  network  is 
evaluated.  Two  outcomes  are  possible: 

1 )  None  of  the  queues  increases  in  size  and  steady  state  is  achieved. 

2)  Some  queues  continue  or  start  to  get  bigger. 

In  case  1 ),  the  learning  block  is  left  in  place  and  its  impact  is  recorded;  in  case  2), 
the  effect  of  the  learning  block  on  the  various  queues  is  recorded.  This  results  in 
four  lists  of  gates: 

a)  gates  whose  queues  continue  to  grow  after  the  application  of  the  tested 
learning  block; 

b)  gates  whose  queues  started  to  grow  immediately  after  the  application  of  the 
tested  learning  block; 

c)  gates  whose  queues  remained  unchanged  by  the  tested  learning  block. 

d)  gates  whose  queues  declined  after  applying  the  tested  learning  block. 

These  four  lists  are  then  forwarded  to  the  self-repair  algorithm,  which  uses  the 
lists  to  extract  information  about  the  topology  of  the  network.  Based  on  its 
analysis  of  current  and  previous  outcomes,  the  algorithm  then  selects  the  next 
learning  block  to  be  applied  to  the  system.  This  analysis  is  based  on  the  notion  of 
"association  degree",  which  is  described  later  in  this  report.  In  this  way, 
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correction  of  faults  becomes  faster  and  more  effective  with  the  progress  of  time, 
as  the  self-repair  algorithm  learns  the  topology  of  the  network  from  its  analysis  of 
the  learning  block  outcomes. 

The  implications  to  our  underlying  applications  are  as  follows.  In  the  case  of  a 
computing  cluster,  assume  that  one  of  the  computing  centers  of  the  cluster  has 
become  inoperable  due  to  damage  or  malfunction,  or  that  a  link  connecting  two 
computing  centers  has  been  damaged.  As  a  result,  the  queue  of  instructions  and 
results  forwarded  to  the  inoperable  computing  center  or  to  the  damaged 
computing  link  starts  growing;  at  some  point,  it  will  exceed  its  warning  bound.  At 
this  point,  a  re-distribution  of  the  flow  of  instructions  between  the  remaining 
computing  centers  of  the  cluster  is  performed  through  the  activation  of  a  learning 
block.  If  the  application  of  the  learning  block  stops  the  growth  of  all  queues  in  the 
network,  then  the  problem  has  been  corrected.  The  results  are  forwarded  to  the 
self-repair  algorithm  to  extract  structural  information  for  future  use.  If  the  learning 
block  does  not  mitigate  the  growth  of  all  queues,  then  it  is  deactivated,  and  the 
results  of  its  impact  are  forwarded  to  the  self-repair  algorithm  for  analysis.  After 
analysis,  the  self-repair  algorithm  suggests  another  learning  block  for  testing, 
based  on  its  analysis  of  the  network  topology.  This  process  improves  the  reaction 
speed  and  the  response  capabilities  of  the  system  through  a  learning  process. 

Similarly,  in  the  case  of  a  digital  communication  network,  the  self-repair 
algorithm  starts  when  one  or  more  of  the  packet  queues  at  a  router  reach  the 
warning  bound.  The  algorithm  then  applies  a  learning  block,  which  consists  of  a 
list  of  gates  whose  transmission  rate  is  reassigned.  The  outcome  of  the  learning 
block  application  is  then  analyzed  by  the  self-repair  algorithm,  with  the  objective 
of  extracting  as  much  structural  information  as  possible  about  the  network.  If 
desirable  results  have  not  been  achieved,  the  learning  block  is  removed,  and 
another  learning  block  is  selected  based  on  the  analysis  of  previous  outcomes.  In 
this  way,  the  self-repair  algorithm  learns  and  becomes  able  to  predict  the 
implications  of  applying  a  learning  block,  and  the  self-repair  process  becomes 
more  and  more  efficient  as  time  goes  by.  Similar  implications  hold  for  multiple 
cooperating  combat  units.  The  learning  algorithm  creates  an  association  between 
each  learning  block  and  its  impact  on  the  status  of  the  network  queues.  The 
algorithm  singles  out  learning  blocks  whose  impact  is  consistent  under  a  given 
set  of  circumstances. 

EXAMPLE  1 .  Referring  to  the  30  node  network  described  earlier  in  this  report, 
consider  the  following  learning  block. 

Learning  Block  A:  increase  the  flow  through  the  west  gate  of  node  12  by  two 
steps;  closes  the  south  gate  of  node  18;  reduces  by  one  step  the  flow  through 
the  north  gate  of  node  27. 

Suppose  that  a  malfunction  occurs  in  the  network,  causing  an  accumulation  in 
the  queues  of  the  following  gates:  the  north  gate  of  node  7;  the  east  gate  of  node 
21 ;  and  the  south  gate  of  node  25.  Assume  that  Learning  Block  A  is  applied  after 
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this  malfunction,  and  that  we  notice  an  improvement  in  the  queues  at  the  east 
gate  of  node  21  and  at  the  south  gate  of  node  25.  From  this  outcome,  we  can 
conclude  that  Learning  Block  A  influences  the  traffic  in  links  that  affect  the  traffic 
flow  to  or  from  the  east  gate  of  node  21  and  at  the  south  gate  of  node  25.  On  the 
other  hand,  it  does  not  seem  to  have  an  influence  on  the  traffic  in  links  that  lead 
to  the  north  gate  of  node  7.  This,  of  course,  is  very  valuable  structural 
information,  which  is  stored  in  the  database  of  the  learning  system.  In  the  future, 
when  the  system  encounters  a  new  malfunction  that  causes  an  increase  in  the 
queues  of  the  east  gate  of  node  21  and  the  south  gate  of  node  25,  the  system 
will  attempt  to  correct  the  problem  by  applying  Learning  Block  A. 

As  the  system  continues  to  operate  and  to  handle  more  and  more  failures  and 
malfunctions  in  the  network,  it  accumulates  more  and  more  relationships  in  its 
database,  relationships  that  associate  each  learning  block  with  the  malfunctions 
it  can  correct.  The  more  malfunctions  the  system  has  experienced,  the  more 
precise  the  associations  between  learning  blocks  and  their  corrective  effects 
become,  and  the  more  capable  the  system  becomes  in  administering  a  quick 
recovery  after  malfunction. 


6.  THE  LEARNING  PROCESS  AND  THE  ASSOCIATION 

DEGREE 


The  learning  process  is  based  on  a  weighting  procedure  that  assigns  greater 
weight  to  learning  blocks  that  correct  a  given  problem  more  often.  Consider  the 
following  example.  To  simplify  the  wording,  we  will  denote  each  buffer  by  a 
number  and  a  letter,  where  the  number  indicates  the  node  number  and  the  letter 
indicates  the  direction  of  the  buffer's  gate.  Recall  that  in  our  simulation  scheme, 
every  node  has  four  gates  -  a  north  gate  (N),  and  east  gate  (E),  a  south  gate  (S), 
and  a  west  gate  (W).  It  is  convenient  to  denote  every  gate  by  a  digit  and  a  letter 
in  the  form  nX,  where  n  is  the  node  number  of  the  gate  and  X  in  one  of  the 
letters  N,  E,  S,  and  W.  In  this  notation,  12E  indicates  gate  east  of  node  12;  20N 
designates  the  north  gate  of  node  20.  Assume  then  that  the  system  went  through 
two  malfunctions: 

Malfunction  1 :  the  queues  of  gates  7S  and  15E  increase; 

Malfunction  2:  the  queues  of  gates  7S  and  23N  increase. 

To  overcome  the  impact  of  these  malfunctions,  the  system  attempts  two  learning 
blocks  -  Learning  Block  B  and  Learning  Block  D,  with  the  outcomes: 

Learning  Block  B  stops  the  queue  growth  in  gate  7S  in  Malfunction  1  and  in 
Malfunction  2;  Learning  Block  D  stops  the  queue  growth  in  gates  7S  and  23N  in 
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Malfunction  2  only,  and  has  no  effect  in  the  case  of  Malfunction  1 . 

These  outcomes  create  a  stronger  association  between  Learning  Block  B  and 
queue  growth  in  gate  7S,  since  Learning  Block  B  impacted  this  queue  in  both 
malfunctions. 

To  formalize  this  process,  we  define  the  novel  concept  of  the  association 
degree.  The  association  degree  assigns  an  integer  to  each  combination  of  a 
building  block  and  a  network  gate,  as  follows.  In  formal  terms,  a  pair  (Y,  nX) 
refers  to  the  learning  block  Y  and  the  gate  nX.  For  example,  the  pair  (C,  9N) 
refers  to  Learning  Block  C  and  the  north  gate  of  node  9. 

The  association  degree  assigns  an  integer  to  each  pair  (Y,  nX).  The  initial 
value  of  this  integer  is  zero,  and  it  is  adjusted  after  analyzing  the  outcome  of 
applying  learning  block  Y  to  a  failure  that  involves  the  queue  at  the  gate  nX. 
The  association  degree  of  (Y,  nX)  is  increased  by  one  after  every  distinct  failure 
pattern  for  which  Learning  Block  Y  reduces  the  queue  at  gate  X  of  node  n.  It 
is  reduced  by  one  if  learning  block  Y  causes  an  increase  in  the  queue  length  at 
gate  X  of  node  n;  and  it  is  left  unchanged  when  learning  block  Y  has  no  effect 
on  the  queue  length  at  gate  X  of  node  n. 

EXAMPLE  2.  Consider  the  situation  described  earlier  in  Example  1 .  Here,  after 
the  two  failure  incidents,  the  association  degree  of  the  pair  (B,  7S)  is  2, 
whereas  the  association  degree  of  the  pair  (D,  7S)  is  1 . 

The  association  degree  is  a  dynamical  quantity  that  changes  and  evolves  as 
the  system  experiences  its  malfunctions.  Its  value  points  to  the  most  likely 
learning  block  for  overcoming  a  given  malfunction.  Intuitively  speaking,  the 
association  degree  is  a  quantitative  summary  of  the  experience  gained  by  the 
system;  a  record  of  what  the  system  has  learned  from  its  experience. 

As  the  system  continues  its  operation  and  encounters  new  malfunctions  and 
failures,  it  examines  the  effects  of  the  malfunction  on  the  queues  of  the  various 
gates  and  generates  a  list  of  the  gates  affected  by  the  malfunction.  Then,  it 
examines  its  current  list  of  association  degrees,  and  chooses  a  learning  block 
that  shows  a  high  association  degree  with  the  gates  affected  by  the  malfunction. 
The  system  then  applies  the  selected  learning  block  and  examines  the  outcome. 
If  the  outcome  is  satisfactory  and  all  queues  have  been  stabilized,  then  the 
system  updates  the  association  degrees  and  continues  its  regular  operation.  If 
the  outcome  is  not  satisfactory,  then  the  system  updates  the  association  degrees 
and  applies  another  learning  block,  choosing  from  among  the  remaining  blocks 
having  highest  association  degree  with  the  affected  queues.  This  process 
continues  until  the  system  compensates  for  the  effects  of  the  latest  malfunction. 
The  association  degree  updates  generated  during  this  process  will  improve  the 
reaction  of  the  system  in  future  malfunctions. 

When  compared  to  genetic  algorithms,  it  is  quite  clear  that  the  learning  block 
approach  provides  faster  and  more  effective  corrections  to  system  malfunctions. 
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When  attempting  a  correction  to  a  malfunction  using  the  learning  block  approach, 
the  system  does  not  just  apply  a  random  remedy;  it  applies  a  remedy  that  has 
worked  well  in  similar  situations  in  the  past.  Therefore,  the  probability  that  the 
applied  learning  block  provides  an  effective  remedy  is  high,  and  this  probability 
increases  as  the  system  gains  more  and  more  experience  correcting 
malfunctions.  The  association  degree  is,  therefore,  a  simple  and  effective  tool  for 
designing  systems  capable  of  learning  and  self-repair. 


7.  ADAPTIVE  CONTROL  OF  LEARNING  SYSTEMS 


This  part  of  our  research  deals  with  the  development  of  a  general  theoretical 
framework  for  the  design  and  implementation  of  learning  and  self-repairing 
systems.  The  system  under  consideration  is  represented  by  an  asynchronous 
sequential  machine.  An  asynchronous  sequential  machine,  sometimes  referred 
to  as  a  clockless  logic  circuit,  represents  the  fastest  computing  machines.  This 
model  captures  a  broad  range  of  applications,  including  distributed  computing 
clusters,  highly  distributed  and  extensively  parallel  computing  systems, 
asynchronous  computer  communication  networks,  and  the  operation  of  multiple 
combat  units. 

More  specifically,  when  considering  distributed  computing  clusters,  each 
computing  center  in  the  cluster  operates  independently,  directing  instructions  and 
computational  data  to  other  members  of  the  cluster  in  an  asynchronous  fashion. 
Data,  results,  and  instructions  are  generated  and  transmitted  by  each  computing 
center  when  required  by  its  internal  processing,  without  regard  to  the  status  of 
other  members  of  the  computing  cluster.  This  indicates  that  the  various  members 
of  a  computing  cluster  are  working  in  an  asynchronous  fashion.  Asynchronous 
operation  has  the  advantage  of  providing  the  highest  possible  speed,  as 
computing  centers  do  not  have  to  wait  for  timing  cues  arriving  from  other 
computing  centers.  Furthermore,  in  cases  of  high  performance  computing,  each 
computing  center  consists  of  a  large  aggregate  of  parallel  processors  working 
asynchronously.  Thus,  an  asynchronous  machine  model  is  the  most  appropriate 
representation  of  this  application  area. 

Regarding  computer  communication  networks,  it  is  widely  known  that 
asynchronous  networks  provide  a  higher  throughput  than  synchronous  networks, 
as  there  is  no  need  to  generate  and  wait  for  timing  pulses;  the  network  operates 
at  the  highest  speed  possible  for  its  components.  Consequently,  an 
asynchronous  machine  model  is  most  suitable  for  the  representation  of  high- 
throughput  computer  communication  networks. 

Finally,  regarding  the  operation  of  multiple  combat  units,  these  most  often 
result  in  asynchronous  operation  as  well,  since  each  unit  engages  the  enemy 
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and  completes  its  tasks  on  its  own  terms,  independently  of  the  momentary 
operation  of  other  units.  Consider,  for  example,  a  concerted  attack  on  a 
distributed  range  of  mountain  caves  in  enemy  territory.  Multiple  attacks  are 
initiated  by  independent  units  on  each  cave  cluster;  obviously,  the  destruction  of 
one  cave  cluster  occurs  independently  of  the  destruction  of  another  cave  cluster, 
as  some  of  the  cave  clusters  may  be  better  protected  or  better  built  than  others. 
Each  unit  then  completes  its  task,  without  synchronizing  the  completion  with 
other  units.  Thus,  the  operation  of  multiple  combat  units  can  be  considered  as  an 
asynchronous  system;  our  objective  is  to  control  this  asynchronous  system  to 
maximize  its  effectiveness. 

To  represent  faults  and  defects  in  the  underlying  application  systems,  we 
allow  our  asynchronous  machine  model  to  have  several  unknown  transitions.  For 
the  unknown  transitions,  the  outcome  is  not  precisely  known;  instead,  several 
options  for  the  transition  are  provided.  Here  is  an  example  of  a  transition  with  two 
options: 

EXAMPLE  3.  Assume  that  when  the  machine  is  in  State  1  and  the  input 
character  u  is  applied,  two  outcome  options  are  possible:  the  outcome  is  either 
a  transition  to  State  2  or  a  transition  to  State  3.  Note  that  the  machine  is 
deterministic,  so  that  once  the  outcome  of  the  transition  has  been  determined 
experimentally,  the  same  outcome  will  be  repeated  in  any  future  activation  of  the 
input  character  u  at  State  1 .  The  situation  can  be  represented  by  the  following 
state  diagram. 


To  represent  an  asynchronous  machine  with  unknown  transitions,  we  use  the 
following  transition  matrix.  Assume  that  the  machine  has  n  states,  say  x1,  x2,  ..., 
xn.  We  build  an  nxn  matrix  in  which  each  row  and  each  column  correspond  to  a 
state.  Thus,  column  1  and  row  1  correspond  to  the  state  x1;  column  2  and 
row  2  correspond  to  the  state  x2;  and  so  on.  Then,  entry  i,  j  of  the  matrix 
consists  of  all  input  characters  that  induce  a  one  step  transition  from  the  state  x1 
to  the  state  xj  (i.e.,  from  the  state  corresponding  to  the  row  to  the  state 
corresponding  to  the  column).  An  input  character  that  induces  an  unknown 
transition  from  a  given  state  will  appear  several  times  in  the  row  corresponding  to 
the  state  of  origin.  Thus,  the  situation  described  in  Example  3  is  represented  by 
the  matrix 
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where  the  squares  □  represent  input  values  not  considered  in  this  example.  As 
we  can  see,  the  input  character  u  appears  twice  in  row  1 :  once  to  represent  a 
possible  transition  from  the  state  x1  to  the  state  x2,  and  once  to  represent  a 
possible  transition  from  the  state  x1  to  the  state  x3,  as  indicated  in  the  earlier 
state  diagram. 

If  there  is  no  transition  possible  from  the  state  x1  to  the  state  xj,  then  we 
enter  the  character  N  in  position  i,j,  where  N  indicates  the  lack  of  a  transition 
(N  is  not  a  character  of  the  input  alphabet). 

An  important  step  in  the  development  of  learning  and  self-repair  algorithms 
for  an  asynchronous  machine  is  the  partition  of  its  state  set  into  subsets  S  :=  {Si, 
S2,  ....  Sk}  of  states  that  can  be  reached  from  each  other  through  a  known 
transition.  Specifically,  for  any  subset  Si,  all  pairs  of  states  within  Si  are 
connected  through  at  least  one  specified  transition.  In  other  words,  transitions 
within  each  one  of  the  sets  Si  are  known  and  specified  at  the  present  time.  On 
the  other  hand,  transitions  from  one  subset  to  another,  say  transitions  from  states 
of  the  subset  Si  to  states  of  the  subset  Sj,  i  +  j,  if  possible,  involve  an  unknown 
(or  unpredictable)  transition.  As  the  learning  and  self-repair  process  revolves  only 
around  the  management  of  unpredictable  transitions,  this  partition  of  the  state 
space  results  in  a  substantial  reduction  of  computational  complexity  of  the 
learning  algorithms. 

The  partition  S  discussed  in  the  previous  paragraph  is  dynamic,  namely,  it 
changes  as  the  system  learns  the  outcome  of  more  and  more  unpredictable 
transitions.  In  fact,  the  learning  process  of  the  machine  is  represented  by  the 
evolution  of  the  partition  S.  The  number  of  elements  in  S  becomes  smaller  and 
smaller  as  the  machine  learns  the  outcome  of  more  and  more  unpredictable 
transitions. 

Recall  that  an  asynchronous  machine  is  described  by  an  equation  of  the  form 
Xk+1  =  f(xk,uk), 

where  xk  is  the  state  of  the  machine  at  step  k,  and  uk  is  the  input  of  the 
machine  at  step  k.  To  represent  this  machine  in  our  theoretical  framework,  we 
have  developed  the  one-step  transition  matrix,  which  was  described  earlier.  Let 
x1,  x2,  ...,  xn  be  the  states  of  the  machine.  The  one-step  transition  matrix  is  then 
an  nxn  matrix  in  which  each  row  and  each  column  correspond  to  a  state.  Thus, 
column  1  and  row  1  correspond  to  the  state  x1;  column  2  and  row  2 
correspond  to  the  state  x2;  and  so  on.  Then,  entry  i,  j  of  the  matrix  consists  of 
all  input  characters  that  induce  a  one  step  transition  from  the  state  x1  to  the  state 
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xj  (i.e.,  from  the  state  corresponding  to  the  row  to  the  state  corresponding  to  the 
column). 

EXAMPLE  4.  Consider  a  machine  with  two  states,  x1  and  x2.  Suppose  the 
machine  accepts  two  input  characters  a  and  b.  Then,  the  one  step  reachability 
matrix 

a  b 
b  a 

indicates  the  following:  the  input  character  a  leaves  the  machine  in  the  state  x1 
and  in  the  state  x2;  the  input  character  b  takes  the  machine  from  the  state  x1 
to  the  state  x2  and  from  the  state  x2  to  the  state  x1. 

Assume  further  that  the  machine  is  affected  by  a  malfunction,  as  a  result  of 
which  some  of  the  transitions  are  not  known.  In  the  above  example,  suppose  we 
don't  know  exactly  what  transitions  the  input  character  a  causes  when  the 
machine  is  in  the  state  x1.  For  example,  suppose,  when  the  machine  is  in  state 
1,  the  input  character  a  either  leaves  the  machine  in  the  state  x1  or  causes  a 
transition  to  the  state  x2.  In  such  case,  the  one-step  transition  matrix  takes  the 
form: 

(a  a,  b  \ 

\  b  a  /' 

The  matrix  indicates  that,  when  the  machine  is  in  the  state  x1,  the  following 
options  are  possible:  when  the  input  character  a  is  applied,  the  resulting 
transition  might  be  either  to  the  state  x1  or  to  the  state  x2;  if  the  character  b  is 
applied,  the  system  moves  to  the  state  x2.  The  remaining  transitions  are  as 
before.  Thus,  the  one-step  transition  matrix  is  a  very  convenient  way  to  represent 
uncertainties  in  the  function  of  an  asynchronous  machine,  uncertainties  created 
by  malfunctions  or  lack  of  precise  information  about  the  machine. 

Dealing  with  one-step  transition  matrices  that  include  uncertainties  requires  a 
special  mathematical  framework,  which  has  been  developed  as  part  of  the 
current  research  project.  This  framework  is  described  in  the  following 
publications,  which  are  attached  as  part  of  this  report. 

8.  PUBLICATIONS 


The  following  publications  report  about  material  developed  in  this  project. 
J.  M.  Yang  and  J.  Hammer 

[2007]  "Counteracting  the  Effects  of  Adversarial  Inputs  on  Asynchronous 
Sequential  Machines",  submitted  for  publication. 
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J.  M.  Yang  and  J.  Hammer 

[2007]  "State  Feedback  Control  of  Asynchronous  Sequential  Machines  with 
Adversarial  Inputs",  submitted  for  publication. 

Copies  of  these  publications  are  provided  in  the  Appendix. 


9.  CONCLUSIONS 


The  present  report  presents  a  summary  of  the  research  conducted  under 
Grant  number  FA8750-06-1 -01 75.  In  the  course  of  this  research,  we  introduced  a 
new  methodology  for  the  design  of  engineering  systems:  the  learning  block 
methodology.  This  methodology  forms  a  theoretical  foundation  for  the 
development  of  equipment  and  systems  capable  of  learning  and  self  repair. 
During  the  course  of  the  research,  other  methodologies  that  could  potentially  be 
used  for  the  design  of  learning  and  self  repairing  systems  were  examined  and 
compared  to  the  learning  block  methodology.  Specifically,  significant  effort  was 
devoted  to  the  examination  of  genetic  algorithms  in  this  context.  The  conclusion 
of  this  effort  was  that  genetic  algorithms  do  not  form  a  suitable  foundation  for 
learning  and  self  repairing  systems. 

The  research  performed  under  this  grant  shows  that  the  learning  block 
methodology  forms  a  solid  and  efficient  foundation  for  the  design  of  learning  and 
self  repairing  systems.  Further  research  in  this  direction  would  explore  the 
application  of  this  methodology  as  a  supervisory  tool  for  specific  critical  systems, 
including  flight  control  systems  and  large-scale  distributed  computing  clusters. 
The  studies  conducted  under  the  project  reported  here  indicate  that  the  learning 
block  methodology  can  help  improve  the  reliability  and  failure-endurance  of  such 
systems  but  at  least  an  order  of  magnitude. 
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APPENDIX:  PUBLICATIONS 


This  appendix  includes  copies  of  the  following  two  publications: 


J.  M.  Yang  and  J.  Hammer 

[2007]  "Counteracting  the  Effects  of  Adversarial  Inputs  on  Asynchronous 
Sequential  Machines",  submitted  for  publication. 

J.  M.  Yang  and  J.  Hammer 

[2007]  "State  Feedback  Control  of  Asynchronous  Sequential  Machines  with 
Adversarial  Inputs",  submitted  for  publication. 


Hard  copies  of  the  publications  are  attached.  The  publications  are  also  included 
in  the  pdf  version  of  this  report. 
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Abstract:  The  problem  of  counteracting  the  effects  of  adversarial  inputs  on  the  operation  of  an 
asynchronous  sequential  machine  is  considered.  The  objective  is  to  build  an  automatic  state-feedback 
controller  that  returns  an  asynchronous  sequential  machine  to  its  original  state,  after  the  machine  has 
undergone  a  state  transition  caused  by  an  adversarial  input.  It  is  shown  that  the  existence  of  such  a 
controller  depends  on  certain  reachability  and  detectability  properties  of  the  affected  machine. 


1.  INTRODUCTION 

In  modern  computing  systems,  one  often  encounters 
unauthorized  and  adversarial  input  agents  that  attempt  to 
interfere  with  the  proper  operation  of  the  system.  We  address 
the  question  of  how  a  computing  system  can  be  made 
immune  to  such  interferences.  Our  approach  is  based  on 
automatic  control:  we  deploy  feedback  controllers  that  take 
corrective  action  whenever  an  adversarial  input  attempts  to 
affect  the  operation  of  the  underlying  computing  system. 

Specifically,  we  consider  asynchronous  sequential  machines 
with  two  inputs:  an  input  for  controlling  the  machine  (the 
control  input),  and  an  input  used  by  an  adversarial  agent  (the 
adversarial  input).  The  control  diagram  is  as  follows. 


Here,  2  is  the  asynchronous  sequential  machine  being 
controlled,  and  C  is  another  asynchronous  machine  serving 
as  the  controller.  The  control  input  of  2  is  u  and  the 
adversarial  input  is  w.  The  controller  C  counteracts  action 
at  w,  making  it  possible  for  the  closed  loop  machine  to 
operate  without  interference.  Necessary  and  sufficient 
conditions  for  the  existence  of  C  are  presented  in  section  6, 
which  also  includes  a  description  of  the  controller's  structure. 
The  closed  loop  machine  of  the  diagram  is  denoted  by 
2c(v,w),  with  v  being  the  external  command  input. 

Recall  that  an  asynchronous  sequential  machine  can  be  in  a 
stable  state  or  in  a  transient  state.  At  a  stable  state,  the 
machine  dwells  indefinitely  until  the  input  character  is 


changed.  Transient  states  are  traversed  by  the  machine  very 
quickly  (ideally,  in  zero  time),  and  are  imperceptible  by  the 
user.  Thus,  when  counteracting  the  effects  of  an  adversarial 
input,  it  is  only  necessary  to  eliminate  the  effects  on  stable 
states.  The  operation  of  the  controller  C  is,  in  fact,  based  on 
this  principle:  when  the  adversarial  input  causes  a  state 
transition  of  2,  the  controller  turns  the  new  state  into  a 
transient  state  of  the  closed  loop  machine,  and  returns  2  to 
the  stable  state  it  had  before  the  interference.  Thus,  2 
resumes  its  original  state  very  quickly  (ideally,  in  zero  time), 
and  the  effect  of  the  adversarial  input  is  eliminated. 

Our  discussion  is  within  the  framework  developed  by 
MURPHY,  GENG,  and  HAMMER  [2002  and  2003],  GENG 
and  HAMMER  [2004  and  2005],  and  VENKATRAMAN  and 
HAMMER  [2006a,  b,  and  c].  Studies  dealing  with  other 
aspects  of  discrete  event  systems  can  be  found  in 
RAMADGE  and  WONHAM  [1987],  HAMMER  [1994, 
1995,  1996a,  1996b,  1997],  DIBENEDETTO,  SALDANHA, 
and  SANGIOVANNI-VINCENTELLI  [1994],  THISTLE 
and  WONHAM  [1994],  BARRETT  and  LAFORTUNE 
[1998],  the  references  cited  in  these  papers,  and  others.  These 
publications  do  not  address  issues  peculiar  to  the  operation  of 
asynchronous  machines,  such  as  the  avoidance  of  critical 
races  and  the  distinction  between  stable  and  transient  states. 

2.  NOTATION  AND  BASICS 

The  machines  we  consider  have  a  control  input  and  an 
adversarial  input,  and  they  provide  their  state  as  output.  Such 
machines  are  represented  by  a  triplet  (AxB,X,f),  where  A 
is  the  control  input  alphabet,  B  is  the  adversarial  input 
alphabet,  X  is  a  set  of  states,  and  f :  XxAxB  — *  X  is  the 
recursion  function.  The  operation  is  according  to 

xk+1  =  f(xk,uk,wk);  (2.1) 

here,  uo,  Ui,  U2,  ...  is  the  control  input  sequence'.  Wo,  Wi,  W2, 
...  is  the  adversarial  input  sequence',  and  x0,  Xi,  x2,  ...  is  the 


sequence  of  the  machine's  states.  The  step  counter  k 
advances  by  one  at  a  change  of  the  machine's  inputs  or  state. 

(2.2)  EXAMPLE.  An  asynchronous  machine  2  with  the 
control  input  alphabet  A  =  {a,  b};  the  adversarial  input 
alphabet  B  =  {a,  |3};  and  the  state  set  X  =  {x1,  x2,  x3}.  The 
recursion  function  f  is  described  by  the  transition  table: 


state 

(a, a) 

(a,P) 

(b,a) 

(b,P) 

x1 

x1 

x1 

x1 

x2 

2 

3 

1 

2 

2 

X 

X 

X 

X 

X 

3 

3 

3 

1 

2 

X 

X 

X 

X 

X 

A  triplet  (x,u,w)  is  a  stable  combination  if  x  =  f(x,u,w),  i.e., 
if  the  state  x  is  a  fixed  point  of  the  function  f.  A  machine 
lingers  at  a  stable  combination  until  an  input  changes.  A 
triplet  (x,u,w)  that  is  not  a  stable  combination  starts  a  chain 
of  transitions  Xj  =  f(x,u,w),  x2  =  f(xj,u,w),  ...  If  this  chain 
terminates,  then  there  is  an  integer  q  >  1  for  which  xq  = 
f(xq,u,w);  then,  (xq,u,w)  is  a  stable  combination  and  xq  is 
the  next  stable  state.  If  this  chain  of  transitions  does  not 
terminate,  then  (x,u,w)  is  part  of  an  infinite  cycle.  In  this 
paper,  we  consider  only  machines  with  no  infinite  cycles; 
thus,  every  triplet  (x,u,w)  has  a  next  stable  state,  as  follows. 

(2.3)  LEMMA.  In  an  asynchronous  machine  without  infinite 
cycles,  there  is  always  a  next  stable  state.  ♦ 

To  prevent  unpredictable  outcomes,  it  is  common  to  enforce 
a  policy  where  only  one  variable  of  an  asynchronous  machine 
is  allowed  change  value  at  any  instant  of  time  (e.g.,  KOHAVI 
[1970]).  This  is  referred  to  as  fundamental  mode  operation. 
All  the  machines  in  this  paper  operate  in  fundamental  mode. 

(2.4)  DEFINITION.  An  asynchronous  machine  2  operates 
in  fundamental  mode  if  its  inputs  change  value  only  when  2 
is  in  a  stable  combination,  and  then  at  most  one  at  a  time.  ♦ 

In  fundamental  mode  operation  of  the  configuration  (1.1), 
only  one  of  the  machines  2  or  C  can  undergo  transitions  at 
any  instant  of  time.  This  leads  us  to: 

(2.5)  PROPOSITION.  Configuration  (1.1)  operates  in 
fundamental  mode  if  and  only  if  the  following  hold: 

(i)  C  is  in  a  stable  combination  while  2  undergoes 
transitions,  and  2  is  in  a  stable  combination  while  C 
undergoes  transitions. 

(ii)  The  inputs  u,  w,  and  v  change  only  while  2  and  C  are 
in  a  stable  combination,  and  then  only  one  at  a  time.  ♦ 

Thus,  the  controller  C  must  be  designed  so  that  (i)  it 
commences  transitions  only  after  verifying  that  2  is  in  a 
stable  combination,  and  (ii)  it  adopts  a  stable  combination 
before  inducing  a  change  in  the  input  of  2.  This  assures  that 
the  closed  loop  system  is  unambiguous  and  deterministic.  As 
transitions  of  asynchronous  machines  are  very  quick  (ideally, 
in  zero  time),  fundamental  mode  operation  is  not  restrictive. 

3.  ADVERSARIAL  INPUTS 

In  general,  the  adversarial  input  character  wk  is  not 


specified;  it  is  only  known  that  it  belongs  to  a  specified 
subset  v  C  B  called  the  adversarial  uncertainty.  To  include 
this  information,  we  write  2  =  (AxB,X,f,v).  Starting  from 
the  initial  state  x0  and  applying  the  control  input  character 
Uo,  the  next  state  of  2  can  be  any  member  of  the  set 

f[x0xu0xv]  :=  Uwgv  f(xo,u0,w)  C  X. 

To  describe  stable  transitions  of  the  machine  2,  let  x'  be  the 
next  stable  state  of  (x,u,w).  The  stable  recursion  function  s 
is  defined  by  setting  s(x,u,w)  :=  x'.  Considering  adversarial 
uncertainty,  all  possible  next  stable  states  form  the  set 

sv(x,u)  :=  s[x,u,v]  =  (s(x,u,w) :  w  £  v}  C  X.  (3.1) 

4.  DETECTABILITY  AND  REACHABILITY 

By  Proposition  2.5,  fundamental  mode  operation  requires  the 
controller  C  to  remain  in  a  stable  combination  until  2  has 
reached  its  next  stable  state.  To  examine  the  conditions  under 
which  such  a  controller  can  be  implemented,  let  w  be  an 
adversarial  input  character.  Assume  that  2  is  in  a  stable 
combination  at  the  state  x,  when  the  control  input  changes  to 
u.  Then,  2  embarks  on  the  string  of  transitions 

0(x,u,w)  :=  {xi  :=  f(x,u,w),  x2  :=  f(xj,u,w),  ...,  xi(u>w)  := 
f(Xi(u,w)-i4Vw)},  (4.1) 

where  X;(UjW)  is  the  next  stable  state.  The  set  of  all  transition 
strings  consistent  with  the  adversarial  uncertainty  v  is: 

0[x,u,v]  :=  (0(x,u,w)  :  w  E  v}.  (4.2) 

The  next  notion  characterizes  our  ability  to  determine  by  state 
feedback  whether  or  not  2  has  reached  its  next  stable  state. 

(4.3)  DEFINITION.  Let  2  be  in  a  stable  combination  with 
the  state  x,  when  the  control  input  character  changes  to  u. 
The  pair  (x,u)  is  detectable  if  it  is  possible  to  determine  by 
state  feedback  whether  2  has  reached  its  next  stable  state.  ♦ 

Here  is  a  test  to  determine  whether  a  pair  is  detectable. 

(4.4)  THEOREM.  Let  2  be  in  a  stable  combination  with  the 
state  x,  when  the  control  input  character  changes  to  u.  Then, 
(i)  and  (ii)  are  equivalent  for  adversarial  uncertainty  e: 

(i)  The  pair  (x,u)  is  detectable. 

(ii)  States  of  the  set  sE(x,u)  appear  only  at  the  end  of  strings 
belonging  to  0[x,u,e]. 

Proof  (sketch).  Consider  a  string  0(x,u,w)  =  {x0,  Xi,  x2,  ..., 
Xj(u,w)} s  where  w  E  e,  and  assume,  by  contradiction,  that  (ii) 
is  not  valid.  Then,  Xj  E  sE(x,u)  for  an  integer  0  <  j  <  i(u,w), 
so  that  (Xj,u,w)  is  a  transient  combination,  since  j  <  i(u,w). 
The  inclusion  Xj  £  sE(x,u)  implies  that  there  is  an  adversarial 
input  character  w'  £  e  for  which  (xj,u,w')  is  a  stable 
combination.  Thus,  Xj  is  a  transient  state  in  (xj,u,w)  while 
being  a  stable  state  in  (xj,u,w'),  so  that,  at  the  state  Xj,  one 
cannot  tell  whether  2  is  in  a  stable  state.  Whence,  (i)  implies 
(ii).  Conversely,  if  every  state  x'  £  sE(x,u)  appears  only  at 
the  end  of  a  string  in  0[x,u,e],  then  x'  is  always  a  stable 
state  of  2,  and  (ii)  implies  (i).  ♦ 
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Thus,  we  can  determine  whether  2  has  reached  its  next 
stable  state  by  checking  if  the  current  state  of  2  is  in  sE(x,u). 
Next,  we  adapt  to  our  present  setting  the  following  notion 
from  MURPHY,  GENG,  and  HAMMER  [2002]  and  [2003], 

(4.5)  DEFINITION.  Let  2  be  an  asynchronous  machine 
with  the  state  set  X  =  {x1,  x2,  ...,  x"}  and  the  stable  transition 
function  s.  Let  w£B  be  an  adversarial  input  character.  The 
one-step  matrix  of  stable  transitions  p(2,w)  of  2  is  an  nxn 
matrix  whose  (i,j)  entry  py(2,w)  consists  of  all  pairs  w|u, 
where  u  is  a  control  input  character  satisfying  xJ  =  s(x‘,u,w); 
if  there  is  no  such  u,  then  py(2,w)  :=  w|N,  where  N  is  a 
character  not  in  A  or  in  B: 

=  jw|N  if  {u  G  A  :  xJ  =  s(x‘,u,w)}  =0, 

Pijl  ’W1  l{w|u:uGA  and  xj  =  s(x‘,u,w)}  otherwise. 


(4.6)  EXAMPLE.  For  the  machine  2  of  Example  2.2, 


p(2,a) 


^{a|a,a|b}  {a|N}  {a|N}\ 
{a|N}  {a|b}  {a|a}  L 
V  {a|b}  {a|N}  {a|a}  > 


To  work  with  p(2,w),  we  use  the  following  projections:  (A+ 
is  the  set  of  all  non-empty  strings  of  characters  of  A.) 


The  matrix  of  m  stable  transitions  of  2  is  defined  by 

R(m,2,w)  :=  vi=1>...>m  p‘(2,w);  (4.8) 

it  characterizes  all  the  transitions  that  can  be  accomplished  in 
m  or  fewer  stable  steps.  Allowing  m  to  grow  indefinitely 
yields  the  extended  matrix  of  stable  transitions  R*(2,w)  := 
V;  >  i  p‘(2,w)  that  characterizes  all  stable  transitions  of  2. 
The  next  statement  resembles  MURPHY,  GENG,  and 
HAMMER  [2003,  Proposition  3.9]. 

(4.9)  LEMMA.  The  following  are  equivalent  for  all  integers 
m>n-l  and  all  i,  j  =  1,  2,  ...,  n. 

(i)  The  entry  Ry(m,2,w)  includes  a  string  w|u  with  u  f  N. 

(ii)  The  entry  Ry(2,w)  includes  a  string  w|u  with  u  f  N.  ♦ 

Thus,  when  m  >  n-1,  the  matrix  R(m,2,w)  characterizes  all 
stable  transitions  of  2  for  the  adversarial  input  character  w. 

(4.10)  DEFINITION.  For  the  adversarial  input  uncertainty  v, 
the  one-step  stable  transitions  matrix  p(2,v)  is  an  nxn 
matrix  with  entries  py(2,v)  :=  vWGv  p(2,w),  i,  j  =  1,  2, ...,  n.  ♦ 

The  matrix  p(2,v)  includes  all  one-step  stable  transitions 
that  are  compatible  with  an  adversarial  input  character  in  v. 


J  w  if  u  7^  N, 

na  w|u  :=  [0  ejse  ’  (onto  adversarial  value),  and 
nc  w|u  :=  u  for  all  w|u  G  B|(A'h  U  N)  (onto  control  value). 

For  two  sets  of  strings  Si,  s2  C  B|(A+  U  N),  we  define  an 
operation  Si  v  s2  akin  to  union  by  using  the  difference  set 

Si  V  s2  :=  [Sj  u  s2]  \  sN, 


where  Sn  is  the  set  of  all  elements  w|N  G  Si  U  s2  for  which 
[w|A+]  0  [s,  U  s2]  f  0,  i.e.,  all  elements  that  appear  both  with 
N  and  non-N  control  input  strings.  Next,  concatenation  of 
strings  Wj|ui,  w2|u2  G  B|(A+  U  N)  is  given  by 


conc(wi|ui,w2|u2) 


Wi|uiu2  if  Wi  =  w2  and  ui,u2^N, 
-Wi|N,  w2|N  otherwise. 


For  subsets  of  strings  Oi,  o2  C  B|(A+  U  N): 


5.  COMPLETE  SETS  OF  STRINGS 

The  next  notion  is  critical  for  feedback  control  (compare  to 
VENKATRAMAN  and  HAMMER  [2006a,  b,  c]). 

(5.1)  DEFINITION.  Let  2  be  an  asynchronous  machine 
with  the  adversarial  uncertainty  v,  and  let  x1,  xJ  be  states  of 
2.  There  is  a  feedback  path  from  x‘  to  xJ  if  there  is  a  state 
feedback  controller  that  takes  2  from  a  stable  combination 
with  x1  to  a  stable  combination  with  xJ  in  fundamental 
mode,  given  only  that  the  adversarial  input  is  within  v.  ♦ 

Below,  we  develop  a  test  to  determine  whether  there  is  a 
feedback  path  from  x1  to  xJ.  If  a  feedback  path  exists,  then 
an  automatic  controller  can  undo  undesirable  transitions  from 
xJ  to  x1.  Note  that,  due  to  fundamental  mode  operation,  the 
adversarial  input  remains  constant  along  a  feedback  path. 


conc(ai,a2)  :=  vSl&Jl>S2ea2  conc(si,s2). 

Now,  define  an  operation  similar  to  matrix  multiplication  for 
two  nxn  matrices  P,  Q  with  entries  in  B|(Ah  U  N): 

(PQ)ij  :=  Vk=i,...,n  conc(Pik,Qkj),  i,  j  =  1,  2,  ...,  n. 

We  can  use  the  powers  pk(2,w)  =  pk~'(2,w)p(2,w),  k  =  1,2, 
...  The  i,  j  entry  of  pk(2,w)  consists  of  all  strings  w|u  that 
take  2  from  a  stable  combination  with  the  state  x1  to  a 
stable  combination  with  the  state  xJ  in  exactly  k  steps;  if 
there  is  no  such  string,  then  u  =  N. 

(4.7)  EXAMPLE.  Continuing  Example  4.6: 

({a|aa,a|ab,a|ba,a|bb}  {a|N}  {a|N}  \ 

{a|ab}  {a|bb}{a|ba,a|aa}  ♦ 

{a|ab,a|ba,a|bb}  {a|N}  {a|aa}  ' 


Adversarial  uncertainty  may  decline  along  a  feedback  path, 
since  the  machine's  response  provides  information  about  the 
adversarial  input.  For  example,  let  the  adversarial  uncertainty 
be  v  =  {w1,  w2},  and  let  s  be  the  stable  recursion  function 
of  2.  Assume  that  2  is  at  a  stable  combination  with  the  state 
x  and  the  control  input  character  u,  when  the  control  input 
changes  to  u'.  We  have  two  options  for  the  next  stable  state: 

x'  :=  s(x,u',w*)  when  the  adversarial  input  character  is  w1; 
x":=  s(x,u',w2)  when  the  adversarial  input  character  is  w2. 

Clearly,  if  x'  f  x",  then  we  can  determine  the  adversarial 
input  character  from  the  next  stable  state,  resolving  the 
uncertainty.  Thus,  the  adversarial  uncertainty  may  decline 
along  a  feedback  path.  To  discuss  the  general  case,  we  need 
some  notation.  Let  S  C  B|A+  be  the  set  of  all  strings  that 
take  2  from  a  stable  combination  with  the  state  x  to  a 
stable  combination  with  the  state  x',  i.e.,  all  strings  w|u  = 


19 


w|u0ui...  E  B|A+  for  which  s(x,u,w)  =  x'.  For  a  string  o  = 
w|u0Ui...Uk  E  S  and  an  integer  q  >  0,  denote 

-=  |w|u0ui...uq  if  q<k, 

°^q  ‘  ^w|u0Ui...Uk  if  q>k. 

The  string  o|q  takes  2  to  a  stable  combination  with  the  state 
xq  :=  s(x,ct|q)  :=  s(x,u0ui...uq,w),  passing  through  the  stable 
states  x0(o)  :=  s(x,o|0),  Xi(o)  :=  s(x,cr|i),  xq(o)  :=  s(x,o|q), 
where  x0(o)  =  x  and  Xk(cr)  =  x'. 

For  a  string  o  =  w|u0Ui...Uk  E  S,  denote 

Up  for  p  =  0,  1,  k, 

Uk  for  all  p  >  k. 

Now,  let  2  be  in  a  stable  combination  with  the  state  z  when 
the  control  input  value  changes  to  u,  and  let  z"  be  the  next 
stable  state  of  2.  The  set  of  all  adversarial  input  characters 
w  E  v  compatible  with  the  transition  s(z,u,w)  =  z"  is 

sa(z,u,z")  :=  {w  E  v  :  s(z,u,w)  =  z"}.  (5.2) 

In  particular,  when  2  is  at  a  stable  combination  with  the 
initial  state  x0  :=  x  and  the  control  input  character  uo,  the 
adversarial  input  character  w  must  satisfy 

w  E  v(x0,Uo)  :=  sa(x0,Uo,Xo)  DvCv  (5.3) 

Thus,  v(x0,Uo)  is  the  true  initial  adversarial  uncertainty.  For 
the  transition  from  x0  to  x'  to  be  possible,  S  must  contain  a 
path  for  each  adversarial  input  character  w  E  v(x0,u0),  i.e., 
we  must  have  v(x0,u0)  C  Ifa  S.  Otherwise,  S  would  be 
incompatible  with  potential  adversarial  inputs.  Further,  let  ui 
be  a  control  input  character,  and  define  the  set 

S(x0,u0ui)  =  {o  E  S  :  o|i  =  w|u0ui  for  some  w  E  B} 

of  all  strings  of  S  whose  control  input  starts  with  UoUi. 
Clearly,  Ui  can  be  a  next  control  character  only  if  it  is 
compatible  with  all  possible  adversarial  inputs,  i.e.,  only  if 

v(x0,Uo)  C  na  S(x0,UoUi). 

Also,  the  pair  (x0,Ui)  must  be  detectable  to  facilitate 
fundamental  mode  operation  of  the  closed  loop  machine, 
since  the  controller  must  react  at  the  next  stable  state. 

Now,  let  Xi  be  the  next  stable  state  reached  with  the  control 
input  character  ui;  the  state  xi  can  be  read  by  the  state 
feedback  controller.  The  fact  that  2  reached  xi  implies  that 
the  adversarial  input  value  w  must  be  within  the  set 

v(x0xi,u0ui)  :=  sa(x0,ui,xi)  f|  v(x0,u0). 

Continuing  in  this  way,  suppose  that  we  are  at  step  p  of  the 
path.  Let  u0Ui...up  be  the  control  input  characters  applied  so 
far,  and  let  x0Xi...xp  be  the  stable  states  2  has  passed  as  a 
result.  The  current  uncertainty  v(x0...xp,u0...up)  C  B  about 
the  adversarial  input  value  is  called  the  residual  adversarial 
uncertainty.  By  iterating  the  earlier  steps,  we  get 

v(x0...xp,u0. . .up)  :=  sa(xp_i,up,xp)  f|  v(x0...xp_1,u0...up_i).  (5.4) 

Now,  let  S(x0Xi...xp,u0Ui...up)  be  the  set  of  all  strings  oES 


having  the  control  inputs  u0Ui...up  and  taking  2  through  the 
stable  states  x0,  Xi,  ...,  xp.  For  a  control  input  character  d, 
denote  by  S(x0Xi...xp,u0Ui...upd)  the  set  of  all  strings  o  E 
S(x0Xi...xp,u0Ui...up)  that  have  the  character  d  as  their  next 
control  input  character.  Then,  the  set  of  all  adversarial  input 
characters  compatible  with  d  is  naS(x0Xi...xp,u0Ui...upd). 
This  set  must  be  compatible  with  the  residual  adversarial 
uncertainty,  namely, 

(5.5)  LEMMA.  The  character  d  E  A  can  be  used  as  the  next 
control  input  character  of  the  machine  2  only  if 
v(x0xi...xp,u0ui...up)  C  na  S(x0x i . . ,xp,u0u i . . ,upd) .  ♦ 

We  show  later  that  the  condition  of  Lemma  5.5  is  critical  for 
the  existence  of  a  controller  that  automatically  counteracts 
adversarial  transitions.  This  leads  us  to  the  following. 

(5.6)  DEFINITION.  Let  S  C  B|A+  be  a  set  of  strings  taking 
2  from  a  stable  combination  with  the  state  x0  to  a  stable 
combination  with  the  state  x'.  Then,  S  is  a  complete  set  if 
the  following  hold  for  all  p  =  0,  1,  ...  and  for  all  control  input 
characters  dEIIp~11  S(x0Xi„.xp,UoUi...up): 

(i)  v(x0x i . . ,xp,u0u i . . .up)  C  na  S(x0Xi...Xp,UoUi...Upd),  and 

(ii)  The  pair  (xp,d)  is  detectable  with  respect  to  the  residual 
adversarial  uncertainty  v(x0Xi...xp,UoUi...up).  ♦ 

A  complete  set  of  strings  can  be  replaced  by  one  of  bounded 
length,  as  follows.  (For  a  set  of  strings  S  C  BIA1,  denote  by 
|S|  the  maximal  length  of  a  control  input  string  in  S.  For  a 
finite  set  Z,  denote  by  #Z  the  number  of  elements  in  Z.) 

(5.7)  LEMMA.  Let  2  be  in  a  stable  combination  at  the  state 
x0  and  the  control  input  value  Uo.  If  there  is  a  complete  set  of 
strings  from  x0  to  x',  then  there  also  is  such  a  complete  set 
S  satisfying  |S|  <  [#v(x0,u0)](n  -  1). 

Proof  (sketch).  Consider  a  string  o  =  w|u  =  w|u0Ui...Uk  E  S 
and  let  x0Xi...Xk  be  the  stable  states  through  which  2  passes 
as  a  result  of  receiving  the  control  input  string  u.  Let  V;  be 
the  residual  uncertainty  at  step  i  of  the  path,  where  v0  := 
v(x0,u0).  Then,  V;  is  a  monotone  declining  function  of  i,  and 
its  minimal  value  is  not  less  than  1.  Divide  the  interval  [0,  k] 
into  segments  of  constant  residual  uncertainty  [0,  ij],  [ii+1, 
i2],  ...,  [im+l,  k],  where  V;  is  constant  over  each  one  of  these 
subintervals.  Since  V;  is  monotonously  declining  and  its 
minimum  cannot  be  less  than  1,  we  get  m  +  1  <  v(x0,Uo),  or 
m  <  v(x0,u0)  -  1. 

Now,  if  any  of  these  subintervals  [i,  i']  has  length  l  >  n, 
then  the  string  of  states  X;Xj+i...Xi.  must  contain  a  repeating 
state,  say  x  :=  xp  =  xr,  where  i  <  p  <  r  <  \+l.  Since  vp  =  vr  by 
construction,  the  control  input  value  up  can  be  replaced  by 
the  control  value  ur  without  disturbing  the  stable 
combination  at  step  p.  Then,  steps  p+1,  p+2,  ...,  r  can  be 
eliminated  from  the  string,  yielding  a  new  segment  with  the 
length  of  l  —  (r  —  p).  This  process  can  be  repeated  again  and 
again,  until  the  length  of  the  resulting  segment  is  less  than  n. 
Applying  the  same  procedure  to  each  one  of  the  segments,  we 
obtain  a  new  path  of  length  not  exceeding  (m+l)(n-l)  = 
[#v(x0,Uo)](n-  1).  ♦ 
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This  brings  us  to  the  main  result  of  this  section. 

(5.8)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous 
machine  and  let  x  and  x'  be  two  states  of  2.  Then,  the 
following  two  statements  are  equivalent. 

(i)  There  is  a  state  feedback  controller  C  that  drives  2  from 
a  stable  combination  with  x  to  a  stable  combination  with  x' 
in  fundamental  mode  operation. 

(ii)  There  is  a  complete  set  of  strings  S  C  BIA1  taking  2 
from  a  stable  combination  with  x  to  a  stable  combination 
with  x'. 

Proof  (sketch).  Assume  that  (ii)  is  valid.  We  use  S  to  build  a 
state  feedback  controller  F(x,x',v)  which,  upon  receiving  the 
external  input  character  v£A,  generates  a  string  of  control 
input  characters  that  takes  2  from  a  stable  combination  with 
x0  :=  x  to  a  stable  combination  with  x'  in  fundamental  mode 
operation.  To  this  end,  let  2  be  in  a  stable  combination  with 
x0,  and  pick  a  control  input  character  ui  £  If1  S.  Since  S  is 
a  complete  set  of  strings,  (x0,Ui)  is  detectable  with  respect 
v(x0,Uo).  Also,  v(x0,u0)  C  na  S(x0,u0ui),  so  Ui  is  compatible 
with  all  possible  adversarial  inputs.  Denote  by  S  the  state 
set  of  F(x,x',v),  by  c|)  the  recursion  function  of  F(x,x',v), 
and  by  q  the  output  function  of  F(x,x',v);  let  be  the 
initial  state  of  F(x,x',v).  We  construct  now  c|>  and  q. 

Upon  a  detectable  transition  of  2  to  x0  with  the  control 
input  character  uo,  the  controller  moves  to  a  stable 
combination  with  its  state  §i,  readying  for  controller  action 
at  the  command  v.  To  this  end,  set 

(J)(5o,(z,t))  :=  for  all  (z,t)  +  (x0,u0), 

<K?o,(xo,Uo))  :=  Ii,  <Kii>(x0,u0))  :=  Ii- 

While  in  the  states  or  |i,  the  controller  applies  to  the 
control  input  of  2  the  external  input  it  receives,  namely 

q(^o,(z,t))  :=  t,  q(|i,(z,t))  :=  t  for  all  (z,t)  £  XxA, 

If,  while  at  |i,  the  controller  F(x,x',v)  receives  the  external 
input  character  v  (the  command  to  start  controller  action),  it 
moves  to  a  stable  combination  with  its  state 

<Kli»(z,t))  :=  §i  for  all  (z,t)  +  (x0,v), 

<Kii>(x0,v))  :=  h,  <Ki2>(xo»v))  :=  h- 

At  the  F(x,x',v)  applies  the  first  character  Ui  of  the 
control  input  string  that  takes  2  to  the  state  x',  so  we  set 

qfej(xo>t))  :=  ui  for  all  t  £  A. 

Since  S  is  a  complete  set,  ui  makes  2  move  to  the  state  Xj 
through  a  detectable  transition.  Whence,  2  is  in  a  stable 
combination  when  it  reaches  Xi.  Upon  detecting  Xj,  the 
controller  moves  to  a  stable  combination  with  its  state  §3: 

<t>(?2,(z,t))  :=  for  all  (z,t)  f  (x,,Ui), 

<Ki2,(xi,v))  :=  §3,  (t>(?3,(xi,v))  :=  §3. 

At  §3,  the  controller  applies  the  next  control  input  value  u2  £ 
n2  S(x0Xi,u0Ui):  q(§3,(xi,t))  :=  u2  for  all  t  £  A.  Since  S  is  a 
complete  set  of  strings,  the  pair  (xj,u2)  is  detectable  for  the 


current  adversarial  uncertainty  v(x0xi,u0ui)  and  v(x0Xi,u0Ui) 
C  na  S(x0xi,u0uiu2).  We  continue  in  this  way  until  the 
controller  F(x,x',v)  generates  the  last  input  character  of  the 
string,  bringing  2  to  x'.  By  Lemma  (5.7),  the  state  x'  can 
be  reached  in  at  most  (n  -  l)[#v(x0,u0)]  steps. 

Conversely,  assume  that  (i)  is  valid,  and  let  F(x,x',v)  be  the 
corresponding  controller.  Let  S  C  B|A+  be  the  set  of  strings 
that  F(x,x',v)  may  generate  for  the  various  possible 
adversarial  input  characters.  To  show  that  S  is  a  complete 
set,  consider  a  control  input  string  u0Ui...up  that  F(x,x',v) 
applies  to  2,  and  let  x0xi...xp  be  the  stable  states  through 
which  2  passes  as  a  result.  By  (5.4),  the  adversarial 
uncertainty  at  step  p  >  0  is  v(x0xi...xp,u0ui...up).  By 
fundamental  mode  operation  of  the  closed  loop  machine,  the 
pair  (xp,up+i)  is  detectable  with  respect  to 
v(x0xi...xp,u0ui...up).  By  Lemma  5.5,  v(x0Xi...xp,u<)Ui...up)  C 
na  S(x0Xi„.xp,UoUi...upd).  Hence,  S  is  a  complete  set.  ♦ 

An  algorithm  for  finding  complete  sets  of  strings  is  described 
in  YANG  and  HAMMER  [2007].  We  turn  now  to  an 
important  definition.  By  (5.3),  we  have  v(x0,u0)  C  v,  so  that 
#v(x0,u0)  <  #v.  Invoking  Lemma  5.7,  we  conclude  that  a 
complete  set  of  strings  S  can  always  be  selected  so  that 

JS|  <  (n  -  l)(#v).  (5.9) 

Recalling  the  matrix  R(m,2,w),  taking  m  =  (n  -  l)(#v),  and 
including  all  adversarial  characters  of  v,  we  arrive  at  the 
following. 

(5.10)  DEFINITION.  The  nxn  matrix 

R(2,v)  :=  vWGv  R((n-l)(#v),2,w) 
is  the  combined  matrix  of  stable  transitions  of  an 
asynchronous  machine  2  with  adversarial  uncertainty  v.  ♦ 

Considering  (5.9),  Lemma  5.7,  and  Theorem  5.8,  we  reach 
the  following  conclusion. 

(5.11)  CORROLARY.  Let  2  be  an  asynchronous  machine 
with  adversarial  uncertainty  v  and  state  set  X  =  {x1,  ...,  xn}. 
The  statements  below  are  equivalent  for  all  i,  j  =  1,  2, ...,  n: 

(i)  There  is  a  state  feedback  controller  that  takes  2  from  a 
stable  combination  with  x‘  to  a  stable  combination  with  xJ 
in  fundamental  mode  operation. 

(ii)  The  i,  j  entry  of  R(2,v)  includes  a  complete  set  of 
strings.  ♦ 

6.  COUNTERACTING  ADVERSARIAL  TRANSITIONS 

Let  2  be  an  asynchronous  machine  at  a  stable  combination 
(x,u,w),  when  the  adversarial  input  character  switches  to  w'. 
An  adversarial  transition  occurs  if  this  switch  causes  2  to 
move  to  a  new  stable  state  x'  f  x.  In  this  section,  we  discuss 
state  feedback  controllers  that  automatically  counteract 
adversarial  transitions.  In  order  to  operate  in  fundamental 
mode,  it  must  be  possible  for  the  controller  to  determine  from 
the  state  of  2  whether  or  not  it  has  reached  the  next  stable 
state.  The  following  is  analogous  to  Definition  4.3. 

(6.1)  DEFINITION.  Let  2  be  at  a  stable  combination  with 
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the  state  x  and  the  control  input  character  u.  The  pair  (x,u) 
is  cidversarially  detectable  if,  after  an  adversarial  transition,  it 
can  be  determined  from  the  current  state  of  2  whether  or  not 
2  has  reached  its  next  stable  state.  ♦ 

Assume  then  that  2  is  at  a  stable  combination  (x,u,w),  when 
the  adversarial  input  character  changes  to  w',  causing  2  to 
move  to  a  stable  combination  with  the  state  x'  f  x.  This 
transition  may  consist  of  a  number  of  intermediate  steps,  say 
x0  :=  x,  xj  :=  f(x0,u,w'),  x2  =  f(xi,u,w'),  ...,  xq  :=  f(xq  i,u,w')  = 
x',  xq  :=  f(xq,u,w').  Similarly  to  (4.1)  and  (4.2),  we  denote 

0(x,u,w')  :=  Xi...xq, 

0[x,u,e]  :=  {0(x,u,w') :  w'  G  e}.  (6'2) 

The  following  has  a  proof  similar  to  that  of  Theorem  4.4. 

(6.3)  THEOREM.  The  two  statements  are  equivalent: 

(i)  The  pair  (x,u)  is  adversarially  detectable  with  respect  to 
the  adversarial  uncertainty  v. 

(ii)  States  of  the  set  sv(x,u)  appear  only  at  the  end  of  strings 
belonging  to  0[x,u,v].  ♦ 

To  guarantee  fundamental  mode  operation  of  the  closed  loop 
machine,  the  use  of  the  machine  2  must  be  restricted  to 
adversarially  detectable  pairs.  This  leads  us  to  the  following 
notion.  (For  a  string  o  =  w|uiu2...uq  G  BxA+,  denote  by  nt  o 
:=  uq  the  last  control  input  character  of  the  string.) 

(6.4)  DEFINITION.  Let  2  be  an  asynchronous  machine 
with  n  states,  adversarial  uncertainty  v,  and  combined 
matrix  of  stable  transitions  R(2,v).  The  reduced  matrix  of 
stable  transitions  Rr(2,v)  is  obtained  by  removing  from  each 
column  j  =  1,  2,  ...,  n  of  R(2,v)  all  strings  o  for  which  the 
pair  (xJ,nt  o)  is  not  adversarially  detectable.  ♦ 

(6.5)  EXAMPLE.  In  Example  2.2,  2  has  only  one 
adversarial  transition:  (x',b,a)  ->  (x',b,|3)  ->  (x2,b,|3).  Then, 
0[x',b,a]  =  x1  and  0[x',b,|3]  =  x2,  so  0[x',b,v]  =  {x1,  x2}. 
Also,  sv(x',u)  ={x’,  x2}  here.  Thus  ( x 1 , b )  is  adversarially 
detectable  by  Theorem  a381,  and  Rr(2,v)  =  R(2,v).  ♦ 

The  set  of  adversarial  input  characters  that  give  rise  to  an 
adversarial  transition  from  a  stable  combination  with  the  pair 
(xs,u)  to  a  stable  combination  with  the  pair  (x‘,u)  is 

v(xs,x‘,u)  :=  sa(xs,u,x‘)  fl  v.  (6.6) 

Here,  a  transition  occurs  if  and  only  if  v(xs,x‘,u)  f  0-  We  can 
state  now  the  main  result  of  this  section;  the  proof  is  similar 
to  that  of  Theorem  5.8. 

(6.7)  THEOREM.  Let  2  be  an  asynchronous  machine  with 
the  state  set  {x1,  x2,  ...,  x"}  and  the  reduced  matrix  of  stable 
transitions  Rr(2,v),  and  let  xs  and  x1  be  states  for  which 
v(xs,x‘,u)  f  0.  Then,  the  following  are  equivalent: 

(i)  There  is  a  state  feedback  controller  that  automatically 
reverses  an  adversarial  transition  from  the  state  xs  to  the 
state  x'  in  fundamental  mode  operation. 

(ii)  The  entry  R[s(2,v)  includes  a  complete  set  of  strings 
with  respect  to  the  adversarial  uncertainty  v(xs,x‘,u).  ♦ 
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1.  INTRODUCTION 


Unauthorized  and  adversarial  input  agents  have,  regretfully,  become  an  unavoidable  feature  of  modern  day 
computing.  These  agents  aim  to  interfere  with  the  proper  operation  of  computing  systems,  often  attempting  to 
subdue  them  to  hostile  objectives.  The  present  paper  addresses  the  question  of  how  a  computing  system  can  be  made 
immune  to  such  attempts  to  interfere  with  its  operation.  The  approach  taken  here  is  based  on  automatic  control. 
Specifically,  we  propose  to  deploy  automatic  controllers  that  continually  monitor  a  computing  system  and  take 
corrective  action  whenever  an  adversarial  input  attempts  to  intercede.  The  paper  presents  necessary  and  sufficient 
conditions  for  the  existence  of  such  controllers;  when  a  controller  exists,  an  algorithm  for  its  design  is  also  provided. 

In  formal  terms,  the  discussion  revolves  around  asynchronous  sequential  machines  that  have  two  input  signals: 
one  input  signal  facilitates  control  of  the  machine,  while  the  other  input  signal  is  used  by  an  adversarial  agent  (or  by 
a  disturbance)  to  interfere  with  the  operation  of  the  machine.  The  control  diagram  is  as  follows. 


Here,  2  is  the  asynchronous  sequential  machine  being  controlled,  and  C  is  another  asynchronous  sequential 
machine  that  serves  as  a  controller.  The  machine  2  has  two  inputs:  u  is  the  control  input  used  to  steer  the  machine 
to  proper  operation;  and  w  is  the  adversarial  input  operated  by  an  adversarial  agent  or  by  a  disturbance.  The 
purpose  of  the  controller  C  is  to  counteract  action  at  the  input  w  and  to  endow  the  closed  loop  machine  with 
desirable  behavior.  Section  7  presents  necessary  and  sufficient  conditions  for  the  existence  of  such  a  controller  C, 
and  section  8  describes  the  structure  of  the  controller.  The  closed  loop  machine  described  by  the  diagram  is  denoted 
by  2c(v,w),  where  v  is  the  external  input  of  the  closed  loop  machine.  In  our  present  discussion,  C  is  a  state 
feedback  controller  -  it  has  access  to  the  state  of  the  machine  2. 

Our  discussion  is  within  the  context  of  model  matching.  Let  2'  be  an  asynchronous  sequential  machine  that 
describes  the  desired  behavior  of  the  closed  loop  system.  We  refer  to  2'  as  the  model.  Of  course,  the  model  2'  is 
not  affected  by  the  adversarial  input  w  -  it  has  no  adversarial  input;  2'  accepts  only  user  commands  as  its  input. 
The  purpose  of  the  controller  C  is  to  drive  the  machine  2  so  that,  from  a  user's  perspective,  the  closed  loop  system 
matches  the  behavior  of  the  model  2',  irrespective  of  actions  taken  at  the  adversarial  input.  In  other  words,  we 
would  like  to  have  2c(»,w)  =  2'(»)  for  all  adversarial  input  values  w.  This  is  the  perturbed  model  matching 
problem. 

Recall  that  an  asynchronous  sequential  machine  has  two  kinds  of  states  :  stable  states,  namely,  states  at  which 
the  machine  dwells  indefinitely  with  its  present  input  value,  and  transient  states  -  states  the  machine  passes 
transiently  along  its  way  to  the  next  stable  state.  Only  stable  states  are  perceivable  by  the  machine's  user;  transient 
states  are  traversed  by  the  machine  very  quickly  (ideally,  in  zero  time),  and  are  imperceptible  to  the  user.  Thus, 
when  eliminating  the  effects  of  an  adversarial  input,  it  is  only  necessary  to  eliminate  the  effects  on  stable  states; 
effects  on  transient  states  are  unnoticeable  and,  therefore,  inconsequential. 

The  ability  to  control  an  asynchronous  machine  2  so  as  to  eliminate  the  effects  of  adversarial  inputs  and  match 
a  specified  model  depends  on  certain  features  of  reachability  and  detectability  introduced  in  section  4.  The  essence 
of  these  features  is  condensed  in  sections  6  and  7  into  a  numerical  matrix  of  zeros  and  ones,  called  a  skeleton  matrix. 
The  skeleton  matrix  characterizes  the  possibilities  of  controlling  the  machine  2  in  the  presence  of  an  adversarial 
input.  More  specifically,  the  skeleton  matrix  characterizes  those  aspects  of  the  performance  of  2  that  can  be 
preserved  by  an  automatic  controller  despite  activity  at  the  adversarial  input.  In  section  7,  we  show  that  the 
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perturbed  model  matching  problem  is  solvable  if  and  only  if  the  skeleton  matrix  satisfies  a  certain  numerical 
inequality  (compare  to  MURPHY,  GENG,  and  HAMMER  [2002  and  2003]). 

The  present  paper  deals  with  the  control  of  asynchronous  sequential  machines  utilizing  the  formalism  of 
MURPHY,  GENG,  and  HAMMER  [2002  and  2003],  GENG  and  HAMMER  [2004  and  2005],  and 
VENKATRAMAN  and  HAMMER  [2006a,  2006b,  and  2006c].  Of  course,  asynchronous  sequential  machines  are  a 
topic  within  the  general  area  of  discrete  mathematics.  To  a  large  extent,  our  terminology  and  notation  follow 
EILENBERG  [1974].  Studies  dealing  with  other  aspects  of  the  control  of  discrete  event  systems  can  be  found  in 
RAMADGE  and  WONHAM  [1987],  HAMMER  [1994,  1995,  1996a,  1996b,  1997],  DIBENEDETTO, 
SALDANHA,  and  SANGIOVANNI-VINCENTELLI  [1994],  THISTLE  and  WONHAM  [1994],  BARRETT  and 
LAFORTUNE  [1998],  the  references  cited  in  these  papers,  and  others.  It  seems,  however,  that  these  publications  do 
not  address  issues  that  are  peculiar  to  the  function  of  asynchronous  machines,  such  as  the  avoidance  of  critical  races 
and  the  distinction  between  stable  states  and  transient  states. 

An  important  aspect  of  the  operation  of  asynchronous  sequential  machines  is  fundamental  mode  operation  (e.g., 
KOHAVI  [1970]).  Under  fundamental  mode  operation,  the  input  variables  of  an  asynchronous  machine  are  kept 
constant  while  the  machine  undergoes  state  transitions.  Fundamental  mode  operation  comes  to  guarantee 
deterministic  behavior.  Indeed,  if  an  input  value  is  changed  while  a  machine  undergoes  state  transitions,  then  the 
state  at  which  the  input  change  occurs  becomes  unpredictable;  this  may  result  in  an  unpredictable  outcome. 

In  the  case  of  configuration  (1.1),  fundamental  mode  operation  means  that  (i)  the  controller  C  must  be  in  a 
stable  state  while  2  undergoes  transitions,  and  (ii)  the  machine  2  must  be  in  a  stable  state  while  C  undergoes 
transitions.  All  systems  considered  in  this  paper  are  designed  to  operate  in  fundamental  mode. 

The  paper  is  organized  as  follows.  Section  2  reviews  and  expands  the  basic  notation  and  framework  of  our 
discussion.  Section  3  introduces  adversarial  inputs  and  examines  some  of  their  potential  effects.  Two  notions  that 
are  critical  to  the  solution  of  the  adversarial  model  matching  problem  -  the  notions  of  reachability  and  detectability  - 
are  discussed  in  section  4.  Section  5  introduces  a  test  that  determines  whether  or  not  an  adversarial  action  can  be 
counteracted.  Necessary  and  sufficient  conditions  for  the  existence  of  a  controller  that  solves  the  perturbed  model 
matching  problem  are  derived  in  sections  6  and  7,  while  the  structure  of  the  controller  is  described  in  section  8.  The 
paper  concludes  in  section  9  with  a  comprehensive  example. 


2.  NOTATION  AND  BASICS 


Let  A  be  a  finite  non-empty  alphabet,  let  A  be  the  set  of  all  finite  strings  of  characters  of  A,  and  let  A+  be 
the  set  of  all  non-empty  strings  in  A  .  To  simplify  our  notation  later,  we  assume  that  the  alphabet  A  does  not 
include  the  digits  0  and  1 .  The  length  |w|  of  a  string  w£A  is  the  number  of  characters  of  w.  For  two  strings 
w,,  w2  G  A  ,  the  concatenation  is  the  string  w  :=  w,w2  obtained  by  appending  w2  to  the  end  of  w,.  A  partial 
function  f :  S,  — »  S2  is  a  function  whose  domain  is  a  subset  of  S^ 

Consider  an  asynchronous  sequential  machine  with  two  inputs:  a  control  input  through  which  the  machine  is 
operated  and  an  adversarial  input  that  attempts  to  interfere  with  the  operation  of  the  machine.  We  represent  such  a 
machine  by  a  sextuple  2  =  (AxB,  Y,X,x0,f,h),  where  A  is  the  control  input  alphabet,  B  is  the  adversarial  input 
alphabet,  Y  is  the  output  alphabet,  X  is  a  set  of  n  states,  and  x0  is  the  initial  state  of  the  machine;  f :  XxAxB  -*  X 
(the  recursion  function)  and  h  :  X  — »  Y  (the  output  function)  are  partial  functions.  The  machine  2  operates 
recursively  according  to 

Xk+1  =  f(xk>Uk>Wk), 

(  ’  yk  =  h(xk),k  =  0,  1,2,  ... 

Here,  u0,  u1;  u2,  ...  is  the  control  input  sequence,  while  Wo,  Wi,  w2,  ...  is  the  adversarial  input  sequence ; 
x0,  x,,  x2,  ...  is  the  sequence  of  the  states  through  which  the  machine  passes,  and  y0,  y,,  y2,  ...  is  the  sequence  of 
output  values.  The  integer  k  represents  the  step  counter;  it  advances  by  one  upon  any  change  of  the  machine's 
inputs  or  state.  Having  selected  the  Moore  representation  for  2,  the  output  function  h  does  not  depend  on  the  input 
variables  u  and  w  (e.g.,  KOHAVI  [1970]). 
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(2.2)  EXAMPLE.  The  following  represents  an  asynchronous  machine  2  with  adversarial  input.  Here,  the  control 
input  alphabet  is  A  =  {a,  b};  the  adversarial  input  alphabet  is  B  =  {a,  |3};  the  state  set  is  X  =  {x1,  x2,  x3}.  We 
assume  here  that  the  state  of  the  machine  is  also  its  output,  so  that  the  output  function  h  is  the  identity  function.  The 
recursion  function  f  of  2  is  described  by  the  following  diagram: 


Alternatively,  the  transition  function  f  can  be  characterized  by  the  following  transition  table. 


(a, a) 

(a,P) 

(b,a) 

(b,P) 

X1 

x1 

x1 

x1 

x2 

2 

3 

1 

2 

2 

X 

X 

X 

X 

X 

3 

3 

3 

1 

2 

X 

X 

X 

X 

X 

A  triplet  (x,u,w)  is  a  valid  combination  of  2  if  it  is  included  in  the  domain  of  the  function  f.  Occasionally,  we 
use  a  single  character  for  the  two  inputs,  as  in  ak  =  (uk,wk),  k  =  0,  1,  2,  ...  The  input  sequence  a0,  ai,  a2, ...  is 
permissible  if  all  pairs  (x0,a0),  (xi,aj),  (x2,a2), ...  are  valid. 


Consider  an  input  string  with  repeated  characters,  say  aaabbcccc.  In  compressed  notation,  the  repetitions  are 
omitted,  so  our  string  becomes  simply  abc.  This  notation  conforms  to  the  way  the  string  is  applied  in  practice  -  the 
first  input  character  is  kept  constant  for  the  first  three  steps,  then  the  second  character  is  kept  constant  for  two  steps, 
and,  finally,  the  last  input  character  is  kept  constant  for  four  steps. 

The  machine  2  is  an  input/state  machine  when  its  output  is  the  state,  namely,  when  Y  =  X  and  h(x)  =  x  for 
all  x£X.  Then,  yk  =  xk  for  all  k  =  0,  1,2,...,  and  the  machine  is  described  by  the  recursion 

(2.3)  2  :  xk+1  =  f(xk,uk,wk). 

An  input/state  machine  is  represented  by  a  quadruple  2  =  (AxB,X,x0,f).  The  present  paper  deals  with  the  control  of 
asynchronous  input/state  machines. 

A  valid  triplet  (x,u,w)  is  a  stable  combination  if  x  =  f(x,u,w),  namely,  if  the  state  x  is  a  fixed  point  of  the 
recursion  function  f.  An  asynchronous  machine  lingers  at  a  stable  combination  until  a  change  occurs  at  one  of  its 
inputs.  A  triplet  (x,u,w)  that  is  not  a  stable  combination  is  a  transient  combination. 

A  transient  triplet  (x,u,w)  initiates  a  chain  of  transitions  x,  =  f(x,u,w),  x2  =  f(x,,u,w),  ...,  where  the  input 
characters  u  and  w  are  kept  fixed  while  the  states  change.  This  chain  of  transitions  may  or  may  not  terminate.  If  it 
terminates,  then  there  is  an  integer  q  >  1  for  which  xq  =  f(xq,u,w),  i.e.,  (xq,u,w)  is  a  stable  combination.  Then,  xq  is 
the  next  stable  state  of  x  with  the  input  pair  (u,w).  If  the  chain  of  transitions  does  not  terminate,  then  the  triplet 
(x,u,w)  is  part  of  an  infinite  cycle.  In  this  paper,  we  restrict  our  attention  to  machines  that  have  no  infinite  cycles. 
Thus,  in  our  case,  every  valid  triple  (x,u,w)  has  a  next  stable  state.  For  future  reference,  it  is  convenient  to  record 
this  fact. 

(2.4)  LEMMA.  In  an  asynchronous  machine  without  infinite  cycles,  every  valid  combination  has  a  next  stable  state. 
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When  operating  an  asynchronous  machine,  one  has  to  be  careful  to  prevent  situations  where  two  or  more 
variables  change  value  at  the  same  time.  A  simultaneous  change  of  two  or  more  variables  may  cause  an 
asynchronous  machine  to  become  unpredictable  (e.g.,  KOHAVI  [1970]).  Thus,  it  is  common  to  enforce  a  policy 
where  only  one  variable  is  allowed  to  change  value  at  any  instant  of  time.  When  this  policy  is  enforced,  the  machine 
2  operates  in  fundamental  mode.  In  practice,  almost  all  asynchronous  machines  are  operated  in  fundamental  mode. 

The  asynchronous  machine  2  of  (2.3)  involves  three  variables:  the  state  x  and  the  input  variables  u  and  w. 

In  fundamental  mode  operation,  not  more  than  one  of  these  variables  can  change  value  at  any  instant  of  time.  This 
leads  to  the  following. 

(2.5)  DEFINITION.  Let  2  =  (AxB,X,f)  be  an  asynchronous  input/state  machine  with  the  two  input  variables  u  and 
w.  The  machine  2  operates  in  fundamental  mode  if  u  and  w  change  values  only  when  2  is  in  a  stable 
combination,  and  then  at  most  one  at  a  time.  ♦ 

All  the  machines  discussed  in  this  paper  operate  in  fundamental  mode.  Note  that  lundamental  mode  operation  is 
not  an  overly  restrictive  requirement.  An  asynchronous  machine  reaches  its  next  stable  combination  very  quickly  - 
ideally,  in  zero  time;  thus,  it  is  rather  unlikely  that  changes  in  u  or  in  w  will  occur  while  2  is  in  transition. 
Similarly,  the  probability  that  the  two  independent  variables  u  and  w  will  change  values  simultaneously  is  zero, 
since  the  two  are  not  synchronized.  Fundamental  mode  operation  is  most  common  in  practice,  and  we  adopt  it  as  our 
mode  of  operation. 

Transitions  from  one  stable  combination  of  2  to  another  are  governed  by  its  stable  recursion  function  s,  which 
is  defined  as  follows.  For  a  valid  triplet  (x,u,w)  of  the  machine  2,  let  x'  be  the  next  stable  state;  the  stable 
recursion  function  s  :  XxAxB  — »  X  is  defined  by  the  assignment  s(x,u,w)  :=  x'.  As  2  has  no  infinite  cycles,  every 
valid  triplet  has  a  next  stable  state,  and  hence  s  is  defined  on  all  valid  triplets.  The  stable  recursion  function  s 
induces  the  stable  state  machine  2|S  of  2,  which  is  an  asynchronous  input/state  machine  given  by  (AxB,X,s).  Here, 
the  stable  recursion  function  s  replaces  the  recursion  function  f  of  2. 

Consider  an  input  string  a  :=  a0ai...am_i,  where  a;  G  AxB.  Writing  in  terms  of  components,  we  have  that  a;  = 
(u;,Wi),  with  U;  G  A  and  W;  G  B,  i  =  0, ...,  m-1.  Assume  that  2  is  in  a  stable  combination  at  the  initial  state  x0, 
when  the  input  string  a  is  applied.  In  fundamental  mode  operation,  the  first  input  value  a0  must  remain  fixed  until 
2  reaches  its  next  stable  state  x,  :=  s(x0,a0)  (ideally,  this  transition  is  completed  in  zero  time).  Anytime  thereafter, 
one  of  the  input  variables  u  or  w  can  change,  providing  the  next  input  pair  oq.  These  input  values  remain  fixed 
until  the  next  stable  state  x2  :=  s(s(x0,a0),a,)  is  reached.  This  process  continues  until  the  last  stable  state  xm  := 
s(...s(s(s(x0,a0),ai),a2)...,am_i)  is  reached.  It  is  convenient  to  use  the  shorthand  notation 

(2.6)  s(x0,a)  =  s(...s(s(s(x0,a0),a1),a2)...,am_i),  a  G  (AxB) 1 . 

When  the  machine  2  =  (AxB,X,f)  is  connected  in  the  closed  loop  configuration  (1.1),  the  output  of  the 
controller  C  serves  as  the  control  input  of  2.  We  therefore  use  A  as  the  output  alphabet  of  C.  Also,  the  input 
variable  v  of  the  controller  C  in  the  diagram  is  really  intended  to  control  the  operation  of  2,  so  we  use  A  as  the 
alphabet  of  v  as  well.  In  addition  to  v,  the  controller  C  accepts  the  state  X  of  2  as  another  input,  so  the  input  set 
of  C  is  AxX.  Letting  H  be  the  state  set,  c(>  the  recursion  function,  and  q  the  output  function  of  the  controller,  we 
can  write  C  =  (AxX,A,H,§0,<t>,'n)>  where  is  the  initial  state  of  C. 

In  fundamental  mode  operation  of  the  configuration  (1.1),  only  one  of  the  asynchronous  machines  2  and  C 
can  undergo  transitions  at  any  instant  of  time.  Adapting  GENG  and  HAMMER  [2004  Proposition  1 .4]  to  our  present 
case,  we  obtain  the  following. 

(2.7)  PROPOSITION.  Let  2  =  (AxB,X,f)  and  C  =  (AxX,A,E,§0,cf>,r|)  be  asynchronous  machines  interconnected  in 
the  configuration  (1.1).  Then,  the  configuration  operates  in  fundamental  mode  if  and  only  if  all  the  following  hold: 

(i)  C  is  in  a  stable  combination  while  2  undergoes  transitions,  and  2  is  in  a  stable  combination  while  C 
undergoes  transitions. 

(ii)  The  inputs  u,  w,  and  v  change  only  while  2  and  C  are  in  a  stable  combination,  and  then  only  one  at  a  time.  ♦ 

In  view  of  Proposition  2.7,  the  controller  C  must  be  designed  so  that  it  commences  transitions  only  after 
verifying  that  2  has  reached  a  stable  combination.  Similarly,  C  must  adopt  a  stable  combination  immediately  prior 


27 


to  inducing  a  change  at  the  input  of  2.  The  controller  C  designed  in  section  8  below  satisfies  these  requirements. 
Fundamental  mode  operation  assures  that  all  transitions  of  the  composite  system  are  unambiguous  and  deterministic. 


3.  ADVERSARIAL  INPUTS  AND  MODEL  MATCHING 


Our  next  objective  is  to  discuss  the  implications  of  the  adversarial  input  on  the  machine  2  =  (AxB,X,x0,f)  of 
(2.3).  The  critical  issue  is,  of  course,  the  fact  that  the  adversarial  input  character  wk  is,  in  general,  not  specified.  The 
only  a-priori  information  available  about  wk  is  that  it  belongs  to  a  specified  subset  vCB  called  the  adversarial 
uncertainty.  Adding  the  adversarial  uncertainty  v  to  the  description  of  2,  we  obtain  the  quintuple 
2=  (AxB,X,x0,f,v). 

When  the  adversarial  uncertainty  v  includes  more  than  one  character,  the  precise  value  of  the  adversarial  input 
is  not  specified;  this  entails  that  the  state  of  the  machine  2  may  not  be  known.  Indeed,  starting  from  the  initial  state 
x0  and  applying  the  initial  control  input  value  Uo,  the  state  of  2  after  the  first  step  can  be  any  member  of  the  set 

f[x0xu0xv]  :=  U  w Gv  f(x0,Uo,w)  C  X. 

When  this  set  contains  more  than  one  state,  one  cannot  predict  the  precise  state  of  2  after  the  first  step.  In  such  case, 
we  are  faced  with  the  control  of  an  asynchronous  machine  in  the  presence  of  uncertainty. 

Recall  that  two  asynchronous  machines  2!  and  22  are  stably  equivalent  if  the  stable  state  machines  2ijS  and 
22|S  are  equivalent  (see  GENG  and  HAMMER  [2004],  [2005]  for  more  details).  Stably  equivalent  machines  have 
equivalent  functionality  and  are  identical  from  a  user's  point  of  view.  We  are  now  ready  to  formally  formulate  the 
main  topic  of  our  discussion. 

(3.1)  THE  PERTURBED  MODEL  MATCHING  PROBLEM.  Let  2  =  (AxB,X,f,v)  be  an  input/state  asynchronous 
machine  with  an  adversarial  input,  and  let  2'  =  (A,X,s')  be  a  stable-state  input/state  machine  with  no  adversarial 
input.  Find  necessary  and  sufficient  conditions  for  the  existence  of  a  controller  C  for  which  the  closed  loop 
machine  2c(v,w)  is  stably  equivalent  to  2'(v)  for  all  w  E  v  and  all  v  E  A.  If  such  a  controller  exists,  derive  an 
algorithm  for  its  design.  ♦ 

(3.2)  EXAMPLE.  A  model  machine  2'.  This  input/state  machine  has  the  control  input  alphabet  A={a,  b}  and  the 
state  set  X={x1,x2,x3}.  It  is  a  stable  state  machine  with  the  stable  transition  function  s'  given  by  the  following 
transition  diagram;  the  model  has  no  adversarial  input. 


2': 


Alternatively,  the  stable  transition  function  s'  of  2'  can  be  described  by  the  following  table  of  transitions. 
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a 

b 

X1 

X1 

x1 

2 

1 

2 

X 

X 

X 

3 

3 

1 

X 

X 

X 

Now,  let  s  be  the  stable  recursion  function  of  the  asynchronous  machine  2  =  (AxB,X,f,v).  Assume  that  2  is 
in  a  stable  combination  with  the  state  x  when  the  control  input  takes  the  value  u.  As  2  has  no  infinite  cycles,  it 
follows  by  Lemma  2.4  that  there  is  a  next  stable  state  x'  of  2.  The  exact  value  of  x'  is  usually  impossible  to 
predict,  since  the  value  w  of  the  adversarial  input  is  not  known.  All  possible  values  of  x'  are  given  by  the  set 

(3.3)  sv(x,u)  :=  s[x,u,v]  =  (s(x,u,w) :  w  E  v}  C  X. 

Letting  P(X)  be  the  family  of  all  subsets  of  the  state  set  X  of  2,  we  thus  obtain  the  function  sv  :  XxA  — »  P(X) 
called  the  perturbed  stable  recursion  function  of  2. 


4.  DETECTABILITY  AND  REACHABILITY 


4.1.  Detectability 


As  we  have  seen  in  Proposition  2.7,  fundamental  mode  operation  of  the  closed  loop  (1.1)  requires  the  controller 
C  to  remain  in  a  stable  combination  until  the  machine  2  has  reached  its  next  stable  state.  Consequently,  it  must  be 
possible  for  C  to  determine  whether  or  not  2  has  reached  its  next  stable  state.  We  examine  now  the  conditions 
under  which  the  latter  is  possible. 

To  this  end,  let  v  be  the  adversarial  uncertainty  of  the  machine  2,  and  let  w  E  v  be  the  adversarial  input 
character.  Assume  that  2  is  in  a  stable  combination  with  the  state  x,  when  the  control  input  variable  takes  the  value 
u.  The  machine  2  embarks  then  on  a  chain  of  state  transitions  given  by 

(4.1)  0(x,u,w)  :=  {X[  :=  f(x,u,w),  x2  :=  f(xbu,w), ...,  xi(u>w)  :=  f(xl(u>w)  i,u,w)}, 

where  xi(UjW)  is  the  stable  state  reached  at  the  end.  Clearly,  the  number  of  steps  i(u,w)  and  the  state  xi(UjW)  depend 
on  the  adversarial  input  character  w  E  v.  According  to  (3.3),  we  must  have  xi(uw)  e  sv(x,u).  The  set 

(4.2)  0[x,u,v]  :=  (0(x,u,w)  :  w  E  v} 

includes  all  state  strings  that  form  chains  of  transitions  consistent  with  the  adversarial  uncertainty  v,  given  that  2 
starts  from  the  state  x  with  the  control  input  character  u.  The  following  statement  shows  that  a  string  in  0[x,u,v] 
includes  no  repeating  states. 

(4.3)  LEMMA.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  no  infinite  cycles.  Let  x  be  a  state  of  2 
and  let  u  be  a  control  input  value.  Then,  a  string  0  E  0[x,u,v]  includes  no  repeating  states. 

Proof.  Consider  a  string  0  =  {x , ,  x2,  ...,  xi(UjW)}  E  0[x,u,v].  First,  if  i(u,w)  =  1,  then  0  includes  only  one  state,  and 
clearly  has  no  repeating  states.  Consider  then  the  case  where  i(u,w)  >  1.  Assume,  by  contradiction,  that  x;  =  xj  for 
some  integers  1  <  i  <  j  <  i(u,w).  Now,  if  j  =  i(u,w),  then  the  triple  (xf,u,w)  =  (xj,u,w)  =  (xi(UjW),u,w)  is  a  stable 
combination.  If  view  of  (4.1),  this  implies  that  0  terminates  at  step  i,  so  that  i  =j  =  i(u,w),  a  contradiction.  Next, 
consider  the  case  where  j  <  i(u,w).  Using  the  recursion  function  of  2,  we  get  that  xi+1  =  f(x;,u,w)  =  f(xj,u,w)  =  Xj+i, 
xi+2  =  f(xi+i,u,w)  =  f(xj+i,u,w)  =  Xj+2,  ...,  which  implies  that  the  machine  2  has  an  infinite  cycle  xi;  xi+i,  xi+2, ...,  Xj, 
xi+1,  xi+2, ...  contradicting  our  assumption.  This  concludes  our  proof.  ♦ 

The  following  notion  underlies  the  controller's  ability  to  determine  whether  or  not  the  controlled  machine  2 
has  reached  its  next  stable  state. 

(4.4)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  input/state  asynchronous  machine,  let  x  EX  be  a  state  of  2,  and 
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let  uG  A  be  a  control  input  character.  Assume  that  2  is  in  a  stable  combination  with  the  state  x,  when  the  control 
input  character  changes  to  u.  Then,  the  pair  (x,u)  is  detectable  if  it  is  possible  to  determine  from  the  state  of  2 
whether  or  not  2  has  reached  its  next  stable  combination.  ♦ 

The  following  is  a  test  that  allows  us  to  ascertain  whether  a  given  pair  (x,u)  is  detectable. 

(4.5)  THEOREM.  Let  2  =  (AxB,X,f)  be  an  input/state  asynchronous  machine  with  an  adversarial  input,  and  let  e 
be  the  adversarial  uncertainty.  Let  xEX  be  a  state  of  2,  let  uGA  be  a  control  input  character,  and  assume  that 

2  is  in  a  stable  combination  with  the  state  x  when  the  control  input  character  changes  to  u.  Then,  in  the  notation  of 
(3.3)  and  (4.2),  the  following  two  statements  are  equivalent: 

(i)  The  pair  (x,u)  is  detectable. 

(ii)  States  of  the  set  sE(x,u)  appear  only  at  the  end  of  strings  belonging  to  0[x,u,e]. 

Proof.  We  use  the  notation  of  (4.1).  Let  w  E  e  be  the  active  adversarial  input  character,  and  let  0(x,u,w)  =  {x0,  x[5 
x2, ...,  Xi(UjW)}  be  the  string  of  transitions  initiated  when  the  control  input  character  switches  to  u.  Assume  that  there 
is  an  integer  0<j<  i(u,w)  for  which  Xj  E  sE(x,u).  Since  j  <  i(u,w),  it  follows  that  (xj,u,w)  is  a  transient 
combination,  as  the  chain  of  transitions  continues  after  j.  By  the  definition  of  the  set  sE(x,u),  the  inclusion  x,  E 
sE(x,u)  implies  that  there  is  an  adversarial  input  character  w'Ge  such  that  the  triple  (xj,u,w')  is  a  stable 
combination.  Summarizing,  the  state  Xj  forms  a  transient  combination  in  the  triple  (xj,u,w)  while  forming  a  stable 
combination  in  the  triple  (xj,u,w').  Consequently,  when  2  is  in  the  state  Xj,  one  cannot  tell  whether  2  is  in  a  stable 
combination  or  not.  In  other  words,  when  (ii)  is  invalid,  so  is  (i). 

Conversely,  assume  that  a  state  x'  E  sE(x,u)  can  appear  only  at  the  end  of  a  string  belonging  to  0(x,u,e).  Then, 
by  the  definition  of  0(x,u,e),  the  machine  2  is  in  a  stable  combination  if  and  only  if  it  is  in  a  state  x'  E  sE(x,u).  In 
other  words,  one  can  determine  from  the  state  of  2  whether  or  not  it  has  reached  its  next  stable  combination,  and 
(x,u)  is  detectable.  Thus,  (ii)  implies  (i),  and  our  proof  concludes.  ♦ 

Thus,  for  a  detectable  pair  (x,u),  a  state  feedback  controller  can  determine  whether  or  not  2  has  reached  its 
next  stable  state  simply  by  checking  whether  the  current  state  of  2  is  a  member  of  sE(x,u).  If  it  is,  then  2  has 
reached  its  next  stable  state;  if  it  is  not,  then  2  has  not  yet  reached  its  next  stable  state.  For  pairs  that  are  not 
detectable,  it  is  not  possible  to  determine  from  the  current  state  whether  2  has  reached  its  next  stable  combination. 

(4.6)  REMARK.  Here,  we  assume  that  the  state  feedback  controller  provides  information  only  about  the  current 
state  of  the  machine  2,  and  does  not  keep  track  of  the  entire  trajectory  of  states  traversed  by  2  on  its  way  to  the 
current  state.  Controllers  that  keep  track  of  the  entire  trajectory  of  2  (i.e.,  controllers  that  record  the  "burst"  of  2) 
have  a  more  complex  structure  and  will  be  discussed  in  a  separate  report.  ♦ 


4.2.  Reachability 


We  turn  now  to  an  examination  of  the  reachability  features  of  asynchronous  machines  with  adversarial  input. 
First,  we  adapt  to  our  present  setting  the  following  notion  from  MURPHY,  GENG,  and  HAMMER  [2002]  and 
[2003], 

(4.7)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  input/state  asynchronous  machine  with  adversarial  input,  having 
the  state  set  X  =  {x1,  x2, ...,  xn}.  Let  s  be  the  stable  transition  function  of  2,  let  u  E  A  be  a  control  input  character, 
and  let  w  E  B  be  an  adversarial  input  character. 

(i)  The  one-step  sample  control  transition  matrix  y(2,w)  of  2  is  an  nxn  matrix  whose  (i,j)  entry  y;j(2,w) 
consists  of  all  control  input  characters  u  E  A  for  which  xJ  =  s(x‘,u,w);  if  there  are  no  such  control  input  characters, 
then  y;j(2,w)  :=  N,  where  N  is  a  character  not  in  A  or  in  B. 

(ii)  The  one-step  sample  adversarial  transition  matrix  ?t(2,u)  of  2  is  an  nxn  matrix  whose  (i,j)  entry  A.;j(2,u) 
consists  of  all  adversarial  input  characters  w  E  B  for  which  xJ  =  s(x‘,u,w);  if  there  are  no  such  adversarial  input 
characters,  then  A.ij(2,u)  :=  N,  where  N  is  a  character  not  in  A  or  in  B.  ♦ 

More  formally,  we  can  write 
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and 


Yij(2,w) 


In  if  {u  E  A  :  xj  =  s(x‘,u,w)}  =  0, 
I{u  E  A  :  xj  =  s(x‘,u,w)}  otherwise, 


In  if  {w  E  v  :  xj  =  s(x‘,u,w)}  =  0, 
l{w  E  v  :  xj  =  s(x‘,u,w)}  otherwise, 

i,j  =  1, ...,  n. 

In  order  to  keep  an  explicit  record  of  the  adversarial  input  character,  we  introduce  the  one-step  matrix  of  stable 
transitions  p(2,w),  where  we  add  the  prefix  w|  to  all  entries  of  y(2,w),  namely, 

Pii(2,w)  :=  {w|v  :  v  0  Yij(2-w)},  i,  j  =  1,  2,  ...,  n. 


(4.8)  EXAMPLE.  Consider  the  machine  2  of  Example  2.2.  The  one-step  matrices  of  stable  transitions  of  2  are 


'  {aa.ab}  {aN}  jaN)  ^ 

'  IPIa}  { P|b}  {P|N}  N 

P(2,a)  = 

{aN}  {ab}  jaa} 

,  P(2,p) = 

{P|a}  {p|b}  {P|N} 

\  {ab}  {a|N}  {a|a}  > 

l  {PIN}  {p|b}  {P|a}  / 

To  work  with  the  entries  of  p(2,w),  we  define  the  projections  na  :  B|(A 1  UN)  (projection  onto  the 
adversarial  input  value)  and  nc  :  B|(A 1  U  N)  — *  (A  UN)  (projection  onto  the  control  input  value)  by  setting 


and 


na  w|u  := 


jw  if  u  f  N, 
10  else, 


IIcw|u:=u  for  all  w|u  E  B|(A 1  U  N). 


Next,  given  two  sets  of  strings  Si,  s2  C  B|(A+  U  N),  we  define  an  operation  Si  v  s2  that  is  akin  to  the  union  of 
the  two  sets,  with  N  being  handled  like  the  empty  set  it  represents.  Specifically,  we  delete  from  the  union  all 
elements  w|N  for  which  w  also  appears  with  a  nonempty  control  input  string.  Using  \  to  denote  the  difference  set, 
we  have 


Si  v  s2  :=  [si  U  s2]  \  sN, 

where  sN  consists  of  all  elements  w|N  E  Sj  U  s2  for  which  [w|A+]  D  [si  U  s2]  f  0. 

Recall  that  concatenation  is  an  operation  that  combines  two  strings  into  one  longer  string.  When  dealing  with 
pairs  of  strings  of  the  form  w|u  £  B|(A4  U  N),  concatenation  operates  only  on  strings  with  the  same  adversarial 
input  character,  concatenating  the  control  inputs  and  leaving  the  adversarial  input  character  unchanged.  Explicitly, 
given  two  strings  Wi|ul5  w2|u2  E  B|(A+  U  N),  set 


conc(wi|ui,w2|u2) 


Wi|uiu2  if  Wi  =  w2  and  both  Ui,u2^N, 
-Wi|N,  w2|N  otherwise. 


For  two  subsets  of  strings  al5  a2  C  B|(A 1  U  N),  the  concatenation  is  defined  by 


conc(Oi,a2)  :=  vSietJljS2ea2  conc(s1,s2). 


Further,  we  define  an  operation  that  resembles  matrix  multiplication.  Let  P  and  Q  be  two  nxn  matrices  with 
entries  in  the  set  B|(A 1  U  N).  The  combination  PQ  is  an  nxn  matrix  with  the  entries 

(PQ)ij  :=  vk=1,...,n  conc(Pik,Qkj),  i,  j  =  1,  2,  ...,  n. 

Using  this  operation,  we  raise  the  one  step  matrix  of  stable  transitions  to  the  power  of  k,  to  get  the  matrix 
Pk(2,w)  =  pk  '(2,w)p(2,w),  k  =  1,  2,  ... 

By  construction,  the  i,  j  entry  of  pk(2,w)  consists  of  all  strings  of  the  form  w|u  that  take  the  machine  2  from  a 
stable  combination  with  the  state  x1  to  a  stable  combination  with  the  state  xJ  in  exactly  k  steps;  if  no  such 
transition  is  possible,  then  u  =  N. 
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P4(2,a)  = 


p4(2,|3)  = 


V 


V 


/ 


(4.9)  EXAMPLE.  Continuing  from  Example  4.8,  we  obtain  (omitting  repeated  characters): 

!  {a|a,a|aba,a|baba,a|ba,ct|abab,a|ab,a|bab,a|b}  {a|N}  {ot|N}  ^ 

{a|aba,a|baba,a|abab,a|bab,ct|ab}  {ct|b}  {a|ba,ct|a} 

{a|baba,a|ba,a|aba,a|abab,a|bab,a|b,a|ab}  {a|N}  {ot|a} 

(  {|3|a,p|baba,|3|aba,(3|ba}  {p|abab,p|bab,p|ab,p|b}  {P|N}  ^ 
{P|a,p|baba,p|aba,p|ba}  {P|abab,P|bab,P|ab,P|b}  {P|N} 
|P|baba,P|ba,P|aba|  {P|abab,P|bab,P|b,P|ab,P|a|  { (3|a) 

The  matrix 


/ 


(4.10)  R(m,2,w)  :=  vi=i . m  p‘(Z,w) 

is  the  matrix  of  m  stable  transitions  of  2.  It  characterizes  all  the  transitions  of  2  that  can  be  accomplished  in  m 
or  fewer  stable  steps.  When  we  allow  m  to  grow  indefinitely,  we  obtain  the  extended  matrix  of  stable  transitions 

R*(2,w)  :=  Vj>!  p‘(2,w), 

which  characterizes  all  stable  transitions  of  the  machine  2. 


Using  the  fact  that  2  has  only  n  states,  an  argument  similar  to  the  one  employed  in  MURPHY,  GENG,  and 
HAMMER  [2003,  Proposition  3.9],  yields  the  following. 

(4.1 1)  LEMMA.  Let  2  =  (AxB,X,f)  be  an  asynchronous  machine  with  n  states  and  an  adversarial  input.  Let  w  be 
an  adversarial  input  character  of  2,  and  let  R(m,2,w)  be  the  matrix  of  m  stable  transitions  of  2.  Then,  the 
following  two  statements  are  equivalent  for  all  integers  m  >  n  1  and  all  i,  j  =  1,  2, ...,  n. 

(i)  The  entry  R,j(m,2,w)  includes  a  string  w|u  with  u^N. 

(ii)  The  entry  R„  (2,w)  includes  a  string  w|u  with  u  f  N.  ♦ 

In  brief  terms,  Lemma  4.1 1  indicates  that  all  stable  transitions  of  the  machine  2  with  the  adversarial  input 
character  w  are  characterized  by  the  matrix  R(m,2,w),  as  long  as  m  >  n— 1. 

Of  course,  the  adversarial  input  character  is  usually  not  known.  To  accommodate  this  fact,  we  introduce  the 
following  notion. 

(4.12)  DEFINITION.  Let  2  be  an  asynchronous  machine  with  the  adversarial  input  uncertainty  v.  The  one- step 
stable  transitions  matrix  p(2,v)  of  2  consists  of  the  entries 

pij(2,v)  :=  vWGV  pjj(2,w),  i,  j  =  1,  2,  ...,  n.  ♦ 

The  matrix  p(2,v)  simply  includes  all  one-step  stable  transitions  that  are  compatible  the  adversarial  uncertainty 
of  2. 


4.3.  State  Feedback 


Recall  that  our  objective  is  to  build  a  state  feedback  controller  C  that  turns  the  closed  loop  machine  2C  of 
diagram  (1.1)  into  a  deterministic  machine  not  affected  by  the  adversarial  input.  The  following  notion  forms  the 
basis  of  our  forthcoming  discussion  (compare  to  VENKATRAMAN  and  HAMMER  [2006b  and  c]). 

(4.13)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  an  adversarial  input,  and  let  x1,  xJ  E 
X  be  two  states  of  2.  We  say  that  there  is  a  feedback  path  from  x1  to  xJ  if  there  is  a  state  feedback  controller  that 
takes  2  from  a  stable  combination  with  x1  to  a  stable  combination  with  xJ  in  fundamental  mode,  given  only  that 
the  adversarial  input  is  within  the  uncertainty  set  v.  ♦ 

In  these  terms,  our  objective  is  to  find  pairs  of  states  of  the  machine  2  that  can  be  connected  by  a  feedback 
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path.  Note  that,  due  to  fundamental  mode  operation,  the  adversarial  input  character  does  not  change  along  a 
feedback  path,  but  its  value  is  not,  in  general,  known.  The  following  is  a  simple  property  of  feedback  paths. 

(4.14)  LEMMA.  Let  2  =  (AxB,X,f)  be  an  asynchronous  machine  with  the  matrix  of  stable  transitions  R(m,2,v). 
Let  x1,  xJ  E  X  be  two  states  of  2,  and  assume  that  there  is  a  feedback  path  from  x1  to  xJ.  If  w  is  a  possible 
adversarial  input  value  along  this  path,  then  w  E  naRij(m,2,v)  for  some  m>l. 

Proof.  Assume  that  there  is  a  feedback  path  from  x1  to  xJ  for  the  adversarial  input  value  w.  Then,  there  is  a  control 
input  string  that  takes  2  from  x1  to  xJ  through  a  string  of  stable  transitions,  while  the  adversarial  input  character  is 
w.  In  view  of  Lemma  4.1 1,  this  implies  that  there  is  a  control  input  string  u  E  A+  such  that  w|u  £  R;j(m,2,v)  for 
some  m  >  1.  Consequently,  w  £  naRjj(m,2,v),  and  our  proof  concludes.  ♦ 

Lemma  (4.14)  provides  a  simple  necessary  condition  for  the  existence  of  a  feedback  path.  We  derive  a 
sufficient  condition  for  the  existence  of  feedback  paths  in  the  next  section. 


5.  COMPLETE  SETS  OF  STRINGS 


Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  an  adversarial  input,  and  let  x‘  and  xJ  be  two  states  of 
2.  In  this  section,  we  develop  a  test  to  determine  whether  there  is  a  feedback  path  from  x1  to  xJ.  Critical  to  this 
development  is  the  uncertainty  about  the  adversarial  input  value.  This  uncertainty  may  vary  along  a  feedback  path 
due  to  the  fact  that  the  controller  accumulates  more  information  about  the  adversarial  input  value. 

As  an  example,  assume  that  v  consists  of  two  characters,  say  v  =  {w1,  w2}.  Then,  initially,  it  is  known  only 
that  the  adversarial  input  value  is  one  of  the  characters  w1  or  w2.  Now,  assume  that  the  control  input  value  is 
changed  to  the  character  u'.  Letting  s  be  the  stable  recursion  function  of  2,  we  have  two  options  for  the  next  stable 
state: 

(i)  x'  :=  s(x,u/,w1)  when  the  adversarial  input  character  is  w1;  and 

(ii)  x":=  s(x,u',w2)  when  the  adversarial  input  character  is  w2. 

Clearly,  if  x'  ^  x",  then  we  can  determine  the  value  of  the  adversarial  input  character  from  the  next  stable  state,  thus 
resolving  the  uncertainty.  On  the  other  hand,  if  x'  =  x",  then  the  outcome  of  this  step  does  not  reduce  the  uncertainty 
about  the  adversarial  input. 

In  summary,  the  uncertainty  about  the  adversarial  input  value  may  be  reduced  as  we  progress  along  a  feedback 
path.  Of  course,  only  the  uncertainty  changes  -  the  adversarial  input  value  itself  is  constant  along  a  feedback  path  in 
fundamental  mode  operation. 

The  adversarial  uncertainty  affects  the  selection  of  the  next  control  input  character,  as  we  now  discuss.  Consider 
the  case  where  the  machine  2  is  at  a  stable  combination  with  the  state  x  and  the  control  input  value  u0,  while 
being  driven  along  a  feedback  path  toward  a  stable  combination  with  the  state  x'.  Let  v0  C  v  be  the  current 
uncertainty  about  the  value  of  the  adversarial  input,  and  let  A  be  the  control  input  alphabet.  Let  S  be  the  set  of  all 
strings  that  take  2  from  its  current  state  x  to  a  stable  combination  with  the  state  x',  i.e.,  all  strings  w|u  =  w|u0Ui  ... 

£  v0|A  ’  for  which  s(x,u,w)  =  x'.  Letting  Ui  be  the  next  control  input  character,  denote  by  S(a)  the  subset  of  all 

strings  of  S  for  which  Ui  =  a,  where  a  £  A  is  a  character  of  the  control  input  alphabet.  Then,  the  set  of  adversarial 
input  characters  that  are  compatible  with  the  control  input  character  a  at  step  1  is  given  by  IIa  S(a). 

Now,  if  v0  (£  na  S(a),  then  the  character  a  cannot  be  used  as  the  control  input  at  step  1,  since  it  is  not 
compatible  with  some  adversarial  input  values  that  may  presently  be  active.  On  the  other  hand,  if  v0  C  IIa  S(a),  then 
a  can  be  applied  as  the  next  control  input  character,  since  it  is  compatible  with  the  information  currently  available 
about  the  adversarial  input  value.  To  conclude,  the  current  level  of  adversarial  uncertainty  v0  impacts  the  selection 
of  the  next  control  input  character. 

For  a  member  o  =  w|u0Ui...Uk  of  S  and  an  integer  q  >  0,  it  is  convenient  to  define  the  truncated  string 
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_  lw|u0ui...uq  if  q<k, 

°^q  '  l-w|u0U|  ...Uk  if  q>k. 

The  set  of  all  truncated  members  of  S  is  denoted  by 

S|q  :=  {o|q  :  a  G  S},  q  =  1,  2, ... 

Recall  that  S  includes  all  strings  that  take  2  from  a  stable  combination  with  the  state  x  to  a  stable 
combination  with  the  state  x'.  Then,  the  string  o|q  takes  2  to  a  stable  combination  with  the  state 

xq  :=  s(x,o|q)  :=  s(x,u0ui...uq,w),  q  =  1,  2,  ... 

The  stable  states  that  2  passes  while  being  driven  by  the  string  c  are  given  by  the  list 
xo(cr)  :=  s(x,a|0),  xi(a)  :=  s(x,a|i), ...,  xk(a)  :=  s(x,a|k), 
where  x0(a)  =  x  and  xk(a)  =  x'. 

For  a  string  a  =  w|u0ui...uk  £  S,  we  define  the  projection  np  :  S  — »  A  which  extracts  the  p-th  control  input 
character  of  a,  i.e., 

ttp  . _  K  for  p  =  0,  1,  ...,  k, 
luk  for  all  p  >  k. 

Next,  assume  that  the  machine  2  is  operated  by  a  state  feedback  controller  C  that  uses  strings  from  the  set  S 
to  drive  2,  while  the  adversarial  input  value  w  is  kept  constant.  As  usual,  there  is  no  direct  information  about  the 
value  of  w.  However,  the  control  input  values  of  2  and  the  states  through  which  2  passes  are  known  to  the 
controller,  as  the  controller  generates  the  input  values  and  reads  the  states  of  2.  This  data  can  be  used  to  reduce  the 
uncertainty  about  the  adversarial  input  value  w,  as  follows. 

Let  x  and  x'  be  two  states  of  2,  let  u  be  a  control  input  string  of  2,  and  let  s  be  the  stable  recursion 
function  of  2.  Assume  that  2  is  in  a  stable  combination  with  the  state  x  when  the  control  input  value  changes  to 
u,  and  let  x"  be  the  next  stable  state  of  2.  Define  the  adversarial  inverse  function  sa  by  setting 

(5.1)  sa(x,u,x")  :=  (w£B:  s(x,u,w)  =  x"}, 

so  that  sa(x,u,x")  is  the  set  of  all  adversarial  input  values  w  £  v  that  are  compatible  with  the  stable  transition 
s(x,u,w)  =  x".  In  particular,  when  2  is  at  a  stable  combination  with  the  initial  state  x0  and  the  control  input  value 
u0,  it  follows  from  (5.1)  that  the  adversarial  input  character  w  must  satisfy 

(5.2)  w  £  v(x0,Uo)  :=  sa(x0,u0,x0)  fj  v. 

Thus,  the  initial  uncertainty  about  the  adversarial  input  value  may,  in  fact,  be  smaller  than  v. 

Recall  that  S  is  the  set  of  all  strings  that  take  2  from  a  stable  combination  with  the  state  x0  :=  x  to  a  stable 
combination  with  the  state  x' .  At  the  initial  step,  the  set  of  all  possible  adversarial  input  values  is  given  by  (5.2). 
Consequently,  the  set  S  must  contain  a  path  for  each  adversarial  input  character  w  £  v(x0,u0),  namely,  we  must 
have  v(x0,Uo)  C  na  S.  Otherwise,  the  set  S  would  be  incompatible  with  some  of  the  potential  adversarial  input 
values. 

Further,  let  Ui  be  a  control  input  character,  and  let  S(x0,UoUi)  be  the  set  of  all  strings  of  S  whose  control  input 
starts  with  u0Ui,  i.e., 

S(x0,u0ui)  =  {o  £  S  :  cr|i  =  w|u0ui  for  some  w  £  B}. 

Clearly,  the  character  Ui  can  be  used  as  the  next  control  input  only  if  it  is  compatible  with  all  possible  adversarial 
input  values,  i.e.,  only  if 

v(x0,Uo)  C  na  S(x0,UoUi). 

As  the  control  input  string  is  generated  by  the  controller  C,  the  pair  (x0,Ui)  must  be  detectable  to  facilitate 
fundamental  mode  operation  of  the  closed  loop  machine. 

Now,  let  xi  be  the  next  stable  state  of  2  reached  with  the  control  input  character  ui.  The  fact  that  2  has 
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reached  the  state  xi  implies  that  the  adversarial  input  value  w  must  have  been  within  the  set 
v(x0xi,uoUi)  :=  sa(x0,ui,xi)  f|  v(x0,u0). 

Continuing  in  this  way,  suppose  that  we  are  at  step  p  of  the  path.  Let  u0Ui...up  be  the  control  input  characters 
applied  so  far  to  2  along  this  path  by  the  controller,  and  let  x0xi...xp  be  the  string  of  stable  states  through  which  2 
has  passed  as  a  result.  Let  v(x0Xi...xp,uoUi...up)  C  B  be  the  current  uncertainty  about  the  adversarial  input  value.  By 
iterating  the  earlier  step,  we  obtain  the  following  conclusion. 

(5.3)  LEMMA.  Let  w  be  the  adversarial  input  character  of  the  asynchronous  machine  2  and  let  p  >  1  be  an 
integer.  Assume  that  the  control  input  string  u0Ui...upEA+  drives  2  through  the  states  x0xi...xp,  where  (xi;Ui,w),  i 
=  0,  1,2, ...,  p,  are  all  stable  combinations.  Then,  w  E  v(x0xi...xp,u0ui...up),  where 

v(x0x1...xp,u0Ui...up)  :=  sa(Xp_i,up,xp)  n  v(x0x1...xp_i,u0Ui...up_i).  ♦ 

Referring  to  Lemma  (5.3),  it  follows  from  (5.2)  that  v(x0xi...xp,u0ui...up)  C  v.  We  call  v(x0xi...xp,u0ui...up)  the 
residual  adversarial  uncertainty  at  step  p. 

Let  us  return  now  to  our  set  of  strings  S  that  take  the  machine  2  from  the  state  x0  :=  x  to  the  state  x'.  Denote 
by  S(x0Xi...xp,UoUi...up)  the  set  of  all  elements  aES  that  satisfy  the  following  conditions: 

(i)  The  control  input  values  are  u0Ui...up;  and 

(ii)  The  machine  2  passes  through  the  states  x0,  xi,  ...,xp. 

For  a  control  input  character  d  E  A,  denote  by  S(x0xi...xp,u0ui...upd)  the  set  of  all  strings  a  E  S(x0Xi...xp,u0Ui...up) 
that  have  the  character  d  in  position  p+1  of  their  control  input  string.  Applying  Lemma  5.3  to  step  p  of  the 
machine  2,  it  follows  that  the  adversarial  input  value  must  be  within  the  set  v(x0xi...xp,u0ui...up).  Also,  the  set  of  all 
adversarial  input  characters  that  appear  with  the  next  control  input  character  d  is  LL,  S(x0x1...xp,u0ui...upd). 
Combining  the  last  two  facts,  we  obtain  the  following. 

(5.4)  LEMMA.  The  character  d  E  A  can  be  used  as  the  next  control  input  character  of  the  machine  2  only  if 
v(x0x1...xp,u0ui...up)  C  na  S(x0x1...xp,u0ui...upd).  ♦ 

The  condition  of  Lemma  5.4  is  critical  to  the  construction  of  a  feedback  controller  that  automatically  takes  a 
machine  2  with  an  adversarial  input  from  one  specified  state  to  another.  In  fact,  we  show  later  that  this  condition 
guaranties  the  existence  of  such  a  controller,  if  it  is  valid  at  every  stable  transition  along  the  way  from  x0  to  x'. 
These  considerations  lead  us  to  the  following. 

(5.5)  DEFINITION.  Let  SCB|AH  be  a  set  of  strings  taking  the  asynchronous  machine  2  from  a  stable 
combination  with  the  state  x0  to  a  stable  combination  with  the  state  x'.  The  set  S  is  complete  if  the  following  two 
conditions  hold  for  all  integers  p  =  0,  1,  2,  ...  and  for  every  control  input  character  d  E  IIP+1  S(x0x1...xp,u0ui...up): 

(i)  v(x0x ! . . ,xp,u0u i . . .up)  C  na  S(x0x1...xp,UoUi...upd),  and 

(ii)  The  pair  (xp,d)  is  detectable  with  respect  to  the  residual  adversarial  uncertainty  v(x0Xi...xp,UoUi...up).  ♦ 

Our  next  objective  is  to  show  that  the  existence  of  a  complete  set  of  strings  is  equivalent  to  the  existence  of  a 
state  feedback  controller.  Shortly  thereafter,  we  present  an  algorithm  for  the  derivation  of  complete  sets  of  strings. 
First,  we  show  that  a  complete  set  of  strings  can  be  replaced  by  a  complete  set  of  strings  of  bounded  length.  For  a  set 
of  strings  S  C  B|A  ,  denote  by  |S|  the  maximal  length  of  a  control  input  string  in  S,  i.e.,  the  maximal  length  of  a 
string  of  the  set  IICS.  For  a  finite  set  Z,  denote  by  #Z  the  number  of  elements  of  Z. 

(5.6)  LEMMA.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  n  states,  and  let  x0  and  x'  be  two  states 
of  2.  Assume  that  2  is  in  a  stable  combination  at  the  state  x0  with  the  control  input  value  u0.  If  there  is  a 
complete  set  of  strings  from  x0  to  x',  then  there  also  is  such  a  complete  set  of  strings  S  satisfying  |S|< 
[#v(x0,Uo)](n  -  1). 

Proof.  Consider  a  string  o  E  S.  Let  u  =  u0Ui...Uk  =  LIca  be  the  control  input  values  of  this  string  and  let  XoXi...Xk  be 
the  string  of  stable  states  through  which  2  passes  as  a  result  of  receiving  the  control  input  string  u.  The  residual 
adversarial  uncertainty  at  the  start  of  the  path  is  v0  :=  v(x0,u0).  Let  V;  be  the  residual  uncertainty  at  step  i  of  the 
path,  and  note  that,  by  definition,  v,  is  a  monotone  declining  function  of  i,  and  its  minimal  value  cannot  be  less  than 
1.  Divide  the  interval  [0,  k]  into  segments  of  constant  residual  uncertainty.  This  results  in  the  set  of  m+1 
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subintervals  I  =  {[0,  ix],  [ii+1,  i2],  [im+l,  k]] ,  where  v,  is  constant  over  each  one  of  these  intervals.  Since  v,  is  a 
monotone  declining  function  and  its  minimum  cannot  be  less  than  1,  we  get  m  +  1  <  #v(x0,u0),  or  m  <  #v(x0,u0)  -  1. 

Now,  if  any  of  the  subintervals  [i,  i']  E  I  has  length  l  >  n,  then  the  string  of  states  xpcj+i.-.x;.  must  contain  a 
repeating  state,  say  x  :=  xp  =  xr,  where  i  <  p  <  r  <  i+l.  Since  vp  =  vr  by  construction,  the  control  input  value  up  can 
be  replaced  by  the  control  value  ur  without  disturbing  the  stable  combination  at  step  p  (recall  that  the  adversarial 
input  value  is  constant  during  the  entire  path).  Then,  steps  p+1,  p+2,  ...,  r  can  be  eliminated  from  the  string, 
resulting  in  a  new  segment  with  the  length  of  l  -  (r  —  p).  This  process  can  be  repeated  again  and  again,  until  the 
length  of  the  resulting  segment  is  less  than  n.  Applying  the  same  procedure  to  each  one  of  the  segments  in  I,  we 
obtain  a  new  path  of  length  not  exceeding  (m+l)(n-l)  <  [#v(x0,u0)](n  -  1).  As  this  bound  is  valid  for  every  segment 
in  I,  our  proof  concludes.  ♦ 

We  have  reached  the  main  result  of  this  section. 

(5.7)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  and  let  x  and  x'  be  two  states  of  2.  Then, 
the  following  two  statements  are  equivalent. 

(i)  There  is  a  state  feedback  controller  C  that  drives  2  from  a  stable  combination  with  x  to  a  stable  combination 
with  x'  in  fundamental  mode  operation. 

(ii)  There  is  a  complete  set  of  strings  SCB|A'  taking  2  from  a  stable  combination  with  x  to  a  stable  combination 
with  x\ 

Proof.  Assume  first  that  (ii)  is  valid.  We  build  a  state  feedback  controller  F(x,x',v)  which,  upon  receiving  the  input 
character  vEA,  generates  a  string  of  control  input  characters  that  takes  2  from  a  stable  combination  with  x0  :=  x 
to  a  stable  combination  with  x'  in  fundamental  mode  operation.  To  this  end,  assume  that  2  is  in  a  stable 
combination  with  the  state  x0,  and  pick  a  control  input  character  ui  £  n1  S.  Due  to  the  fact  that  S  is  a  complete  set 
of  strings,  the  pair  (x0,Ui)  is  detectable  with  respect  to  the  adversarial  uncertainty  v(x0,u0).  In  addition,  v(x0,u0)  C 
na  S(x0,u0ui),  so  that  the  input  character  ui  is  compatible  with  every  possible  adversarial  input  value. 

Now,  let  H  be  the  state  set  of  the  controller  F(x,x',v).  The  recursion  function  <|)  of  F(x,x',v)  has  three 
variables:  the  state  of  F(x,x',v),  the  state  of  2,  and  the  external  control  input,  i.e.,  <|) :  ExXxA  -*  E.  Denote  by  r|  : 
ExXxA  -*  A  the  output  function  of  F(x,x',v),  and  let  |0  be  the  initial  state  of  F(x,x',v).  We  construct  next  the 
functions  4>  and  q. 

Upon  encountering  a  detectable  transition  of  2  to  the  state  x0  with  the  control  input  value  u0,  the  controller 
moves  to  a  stable  combination  with  the  state  This  transition  prepares  the  controller  to  generate  the  input  string 
that  will  take  2  to  the  state  x',  when  commanded  to  do  so;  it  is  accomplished  by  setting 

■=  %  for  a11  C2’1)  +  (xo,u0), 

(|>(^o,(xo,Uo))  :=ii, 

<Ki1,(x0,u0))  :=ii- 

While  in  its  initial  state  §0  or  in  the  state  2ji,  the  controller  applies  to  the  control  input  of  2  the  external  input 
character  it  receives,  namely 

q(io,(z,t))  ~t  for  all  (z,t)GXxA, 
q(|j,(z,t))  :=  t  for  all  (z,t)  £  XxA, 

so  that  F(x,x',v)  is  transparent  in  these  states. 

Suppose  now  that,  while  F(x,x',v)  is  in  the  state  §1;  it  receives  the  external  input  character  v  £  A.  This  is  the 
command  for  the  controller  to  start  a  string  of  transitions  taking  2  from  its  current  stable  combination  with  the  state 
x0  to  a  stable  combination  with  the  state  x'.  Upon  receiving  the  external  input  value  v,  the  controller  F(x,x',v) 
moves  to  a  stable  combination  with  the  state  |2,  namely, 

<K?i,(z>t))  :=  for  all  (z,t)  +  (x0,v), 

<Kir, (x0,v))  :=  h, 

<Ki2,(xo,v))  :=  h- 
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When  reaching  the  state  §2,  the  controller  applies  to  the  control  input  of  2  the  first  character  of  the  control  input 
string  u1U2--.Uk  G  nc  S  that  ultimately  takes  2  to  the  state  x';  to  this  end,  set 

T)(i2>(x0>t))  :=  Uj  for  all  t  G  A. 

The  control  input  character  Ui  causes  2  to  move  to  the  state  xi  through  a  detectable  transition,  since  S  was  a 
complete  set  of  strings.  For  the  same  reason,  Ui  is  compatible  with  every  adversarial  input  character  in  the  residual 
adversarial  uncertainty  v(x0,Uo).  As  the  transition  was  detectable,  2  is  in  a  stable  combination  when  it  reaches  the 
state  Xj.  When  the  controller  detects  the  state  xi  of  2,  it  moves  to  a  stable  combination  with  the  state  §3,  namely, 

<Ki2,(z,t))  :=  h  for  all  (z,t)  +  (x^Ui), 

<Ki2,(x!>v))  :=  |3, 

<Ki3,(xl5v))  :=  |3- 

When  reaching  the  state  |3,  the  controller  applies  to  2  the  next  control  input  value  U2.  Since  S  is  a  complete  set 
of  strings,  the  pair  (xi,U2)  is  detectable  for  the  current  adversarial  uncertainty  v(x0xi,uoUi).  Also,  v(x0xi,uoUi)  C  na 
S(x0xi,uoUiU2),  so  that  U2  is  compatible  with  every  possible  adversarial  input  value.  We  then  build  the  controller 
output  function  accordingly: 

r](|3,(xi,t))  :=u2  for  all  tGA. 

Continuing  in  this  manner,  assume  that  the  controller  F(x,x',v)  has  so  far  generated  the  control  input  string 
u0Ui... Up,  taking  2  through  the  states  x0xi...xp;  here,  p  is  an  integer  between  1  and  k.  Since  S  was  a  complete  set 
of  strings,  the  transition  to  the  state  xp  was  a  detectable  transition;  consequently,  2  is  in  a  stable  combination  when 
it  reaches  the  state  xp.  Upon  detecting  the  state  xp,  the  controller  F(x,x',v)  moves  to  a  stable  combination  with  the 
state  §p+2,  namely, 

<Kip+i’(z’t))  :=  ?p+i  for  all  (z,t)  +  (xp,up), 

<Kip+i>(xP>v)) :=  iP+2, 

‘K?p+2’(XP’V))  :=  ^+2- 

Now,  select  any  control  input  value  up+i  G  Ifp+1  S(x0Xi...xp,u0Ui...up).  Due  to  the  fact  that  S  is  a  complete  set  of 
strings,  the  pair  (xp,up+i)  is  detectable  with  respect  to  the  adversarial  uncertainty  v(x0xi...xp,u0ui...up).  Also, 
v(x0Xi...xp,UoUi...up)  C  na  S(x0xi...xp,u0ui...upup+i),  so  that  the  input  character  upH  1  is  compatible  with  any 
adversarial  character  in  v(x0Xi...xp,UoUi...up).  Upon  reaching  the  state  |p+2,  the  controller  applies  to  2  the  control 
input  character  up+i,  namely, 

Tl(ip+2’(xP,t))  :=  Vl  fora11  teA- 

This  construction  is  repeated  for  p  =  1,  2,  ...,  until  the  machine  2  reaches  the  state  x'.  In  view  of  our 
construction,  the  resulting  controller  F(x,x',v)  satisfies  condition  (i)  of  the  Theorem.  Note  that,  by  Lemma  5.6,  the 
state  x'  can  always  be  reached  at  a  step  k  <  (n  -  l)[#v(x0,u0)],  where  n  is  the  number  of  states  of  the  machine  2. 

Conversely,  assume  that  condition  (i)  is  valid.  Let  F(x,x',v)  be  a  controller  which,  upon  receiving  the  input 
character  v  G  A,  takes  2  from  a  stable  combination  with  the  state  x0  :=  x  to  a  stable  combination  with  the  state  x'. 
Assume  that  2  is  in  a  stable  combination  with  the  state  x0  and  the  control  input  value  u0,  when  the  controller 
input  changes  to  the  character  v.  The  initial  uncertainty  about  the  adversarial  input  character  of  2  is  then  v(x0,Uo). 
Let  SCB|A'  be  the  set  of  strings  the  controller  F(x,x',v)  can  generate;  the  specific  string  used  by  F(x,x',v)  in 
each  case  depends  on  the  information  it  extracts  about  the  adversarial  input  value.  To  prove  that  (i)  implies  (ii),  we 
need  to  show  that  S  is  a  complete  set  of  strings.  Recall  that,  due  to  fundamental  mode  operation,  the  adversarial 
input  character  w,  although  possibly  unknown,  remains  constant  until  2  reaches  the  stable  state  x'. 

To  show  that  S  is  a  complete  set  of  strings,  consider  step  p>0  of  a  string  of  control  input  characters  u0ui...up 
applied  by  F(x,x',v)  to  2.  Denote  by  x0xi...xp  the  stable  states  through  which  2  has  passed  as  a  result  of  this 
string;  here,  xp  is  the  current  stable  state  of  2.  By  Lemma  5.3,  the  residual  adversarial  uncertainty  at  this  point  is 
v(x0Xi...xp,UoUi...up).  Let  dGA  be  the  next  control  input  character  that  F  generates  for  2.  Then,  by  fundamental 
mode  operation  of  the  closed  loop  machine,  the  pair  (xp,d)  is  detectable  with  respect  to  the  residual  uncertainty 
v(x0xi...xp,u0ui...up).  Also,  in  view  of  Lemma  5.4,  we  have  v(x0xi...xp,u0ui...up)  C  na  S(x0Xi...xp,u0Ui...upd).  As  this 
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is  true  for  all  p  >0,  the  requirements  of  Definition  5.5  are  met,  and  S  is  a  complete  set  of  strings.  Thus,  (i)  implies 
(ii),  and  our  proof  concludes.  ♦ 

In  view  of  Theorem  5.7,  finding  a  complete  set  of  strings  is  the  critical  step  in  the  process  of  designing  a 
controller  for  an  asynchronous  machine  with  adversarial  input.  The  following  algorithm  derives  such  a  set  of  strings. 

(5.8)  ALGORITHM.  Derivation  of  a  complete  set  of  strings.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine 
with  adversarial  input.  Let  SCB|A'  be  a  set  of  strings  that  take  2  from  a  stable  combination  with  the  state  x0 
and  the  control  input  value  Uo  to  a  stable  combination  with  the  state  x'.  Consider  the  case  where  2  is  at  step  j  of 
an  input  string  from  S,  having  received  the  control  inputs  u0Ui...Uj  and  having  passed  through  the  stable  states 
x0Xi„.Xj.  The  residual  adversarial  uncertainty  is  v(x0Xi...Xj,u0Ui...Uj). 

Step  0.  Set  j  :=  0. 

Step  1.  If  v(x0Xi...Xj,u0Ui...Uj)  na  S(x0Xi...Xj,u0Ui...Uj),  then  S  does  not  include  a  complete  set  of  strings.  Set  <1>  := 

0  and  terminate  the  algorithm.  Otherwise,  continue  to  Step  2. 

Step  2.  Let  Si  be  the  set  of  all  strings  a  E  S(x0Xi...Xj,u0ui...Uj)  for  which  na  a  v(x0Xi...Xj,u0Ui...Uj).  If  Si  ^  0, 
then  replace  S  by  the  difference  set  S  \  Sj  and  go  to  Step  0.  If  Si  =  0,  continue  to  Step  3. 

Step  3:  Let  S2  be  the  set  of  all  strings  a  €E  S(x0Xi...Xj,UoUi...Uj)  for  which  the  pair  (Xj,Hl+l  a)  is  not  detectable  with 
respect  to  the  residual  uncertainty  v(x0Xi...Xj,u0Ui...Uj).  If  S2  ^  0,  then  replace  S  by  the  difference  set  S  \  S2  and 
go  to  Step  0.  If  S2  =  0,  continue  to  Step  4. 

Step  4.  Let  S3  be  the  set  of  all  strings  a  E  S(x0Xi...Xj,u0Ui...Uj)  for  which  v(x0Xi...Xj,u0Ui...Uj)  na 
S(x0Xi...Xj,u0Ui...UjnJ  1 1  a).  If  S3  ^  0,  then  replace  S  by  the  difference  set  S  \  S3  and  go  to  Step  0.  If  S3  =  0, 
continue  to  Step  5. 

Step  5.  Let  q  be  the  length  of  the  longest  string  in  S.  If  j  =  q,  then  set  <I>  :=  S  and  terminate  the  algorithm. 
Otherwise,  replace  j  by  j+1  and  go  to  Step  1.  ♦ 

The  outcome  of  the  Algorithm  5.8  is  a  set  of  strings  <I>  C  B|A  ' .  If  <E>  is  not  the  empty  set,  then,  according  to  the 
next  statement,  it  forms  a  complete  set  of  strings. 

(5.9)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  adversarial  input,  let  S  C  B|A 1  be  a  set 
of  strings  all  having  the  same  initial  control  input  character,  and  let  <I>  be  the  outcome  of  Algorithm  5.8.  Then, 

(i)  O  is  not  empty  if  and  only  if  S  contains  a  complete  set  of  strings,  and 

(ii)  If  4>  is  not  empty,  then  it  forms  a  complete  set  of  strings  included  in  S. 

Proof.  Assume  that  is  not  empty.  Then,  an  examination  of  Step  3  of  Algorithm  5.8  shows  that  <I>  satisfies 
condition  (i)  of  Definition  5.5,  while  an  examination  of  Step  4  of  the  Algorithm  shows  that  O  satisfies  condition  (ii) 
of  Definition  5.5.  A  slight  reflection  on  the  flow  of  Algorithm  5.8  leads  then  to  the  conclusion  that  O  is  a  subset  of 
S,  and  that  it  is  not  empty  if  and  only  if  S  contains  a  complete  set  of  strings.  ♦ 


6.  SKELETON  MATRICES 


We  turn  now  to  the  definition  of  one  of  the  main  notions  of  our  present  discussion.  Let  2  =  (AxB,X,f,v)  be  an 
input/state  asynchronous  sequential  machine  with  adversarial  input,  having  the  state  set  X  =  {x1,  ...,  x"}  with  n 
states.  In  view  of  (5.2),  the  initial  adversarial  uncertainty  always  satisfies  v(x0,u0)  C  v,  so  we  always  have  #v(x0,u0) 
<  #v.  Invoking  Lemma  5.6,  we  conclude  that  a  complete  set  of  strings  S  for  the  machine  2  can  always  be  selected 
so  that  its  length  satisfies 

(6.1)  |S|  <  (n  —  l)(#v). 

Recall  that,  by  (4.10),  the  i,  j  entry  of  the  matrix  R(m,2,w)  includes  all  the  control  input  strings  that  take  2 
from  a  stable  combination  with  the  state  x1  to  a  stable  combination  with  the  state  xJ  through  a  string  of  m  or 
fewer  stable  transitions,  while  the  adversarial  input  character  is  w.  At  this  point,  it  is  convenient  to  introduce  the 
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matrix 


(6.2)  R(2,v)  :=  vw(EV  R((n-l)(#v),2,w). 

(6.3)  DEFINITION.  The  matrix  R(2,v)  is  the  combined  matrix  of  stable  transitions  of  the  asynchronous  machine  2 
with  the  adversarial  uncertainty  v.  ♦ 

(6.4)  EXAMPLE.  Continuing  with  our  analysis  of  the  machine  2  of  Example  2.2,  we  let  the  adversarial  uncertainty 
be  v  =  {a,  |3},  so  that  #v  =  2.  As  we  have  three  states  in  this  case,  n=  3,  and  (n  -  l)(#v )  =  4. 

R(2,v)  =  p(2,a)  v  p(2,P)  v  p2(2,a)  v  p2(2,P)  v  p3(2,a)  v  p3(2,|3)  v  p4(2,a)  v  p4(2,P)  = 

|a|a,a|aba,a|baba,a|ba,a|abab,a|ab,a|bab,a|b,p|a,p|baba,p|aba,p|ba|  ||3|abab,|3|bab,|3|ab,|3|b}  |a|N,|3|N} ' 

{a|baba,a|aba,a|abab,a|bab,a|ab,p|a,p|baba,p|aba,p|ba}  |a|b,P|abab,P|bab,P|ab,P|b}  |a|ba,a|a}  ♦ 

{a|baba,a|ba,a|aba,a|abab,a|bab,a|b,a|ab,p|baba,p|ba,p|aba}  {|3|abab,|3|bab,|3|b,|3|ab,p|a}  |a|a,p|a}  / 

Considering  (6.1),  Lemma  5.6,  and  Theorem  5.7,  we  reach  the  following  conclusion. 

(6.5)  CORROLARY.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  adversarial  input,  having  the  state  set 
X  =  {x1,  ...,  xn)  and  the  combined  matrix  of  stable  transitions  R(2,v).  Then,  the  following  two  statements  are 
equivalent  for  all  i,  j  =  1,  2, ...,  n. 

(i)  There  is  a  state  feedback  controller  that  takes  2  from  a  stable  combination  with  x1  to  a  stable  combination  with 
xJ  in  fundamental  mode  operation. 

(ii)  The  i,  j  entry  of  R(2,v)  includes  a  complete  set  of  strings.  ♦ 

In  view  of  Corollary  6.5,  it  is  easy  to  determine  whether  or  not  there  is  a  state  feedback  controller  that  takes  the 
machine  2  from  a  stable  combination  with  the  state  x1  to  a  stable  combination  with  the  state  xJ  in  fundamental 
mode  operation:  all  we  have  to  do  is  apply  Algorithm  5.8  to  the  entry  R;j(2,v).  Then,  such  a  controller  exists  if  and 
only  if  the  outcome  of  Algorithm  5.8  is  a  non  empty  set.  This  set  can  then  be  used  to  construct  an  appropriate 
controller  by  following  the  proof  of  Theorem  5.7.  In  this  way,  we  arrive  at  the  following  notion. 

(6.6)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  adversarial  input,  having  n  states  and 
the  combined  matrix  of  stable  transitions  R(2,v).  The  complete  matrix  of  stable  transitions  91(2, v)  of  2  is  an  nxn 
matrix  defined  as  follows  for  each  i,  j  G  {1,  2, ...,  n] :  the  entry  91^(2, v)  is  a  complete  set  of  strings  included  in  the 
entry  R;j(2,v);  if  there  is  no  such  complete  set,  then  91^(2, v)  :=  N.  ♦ 

(6.7)  EXAMPLE.  Applying  Algorithm  5.8  on  the  entries  of  the  matrix  of  stable  transitions  derived  in  Example  6.4, 
we  obtain 

1  {a|a,p|a}  N  N  \ 

91(2, v)  =  {a|ab,p|a,p|ba}  {a|b,p|ab,p|b}  N  ♦ 

'  |a|ab,a|ba,a|b,p|ba}  N  |a|a,p|a|  ' 

In  view  of  Theorem  5.7,  the  following  is  true. 

(6.8)  COROLLARY.  Let  2  be  an  asynchronous  sequential  machine  with  the  state  set  {x1,  ...,  xn}  and  the 
adversarial  uncertainty  v.  Let  9i(2,v)  be  the  complete  matrix  of  stable  transitions  of  2.  Then,  the  following  two 
statements  are  equivalent  for  all  i,  j  G  {1,  ...,  n} : 

(i)  There  is  a  state  feedback  controller  that  takes  2  from  a  stable  combination  with  the  state  x1  to  a  stable 
combination  with  the  state  xJ  in  fundamental  mode  operation. 

(ii)  9ty(2,v)  f  N.  ♦ 

We  can  now  generalize  the  notion  of  the  skeleton  matrix  (MURPHY,  GENG,  and  HAMMER  [2002  and  2003]) 
to  asynchronous  machines  with  adversarial  inputs. 

(6.9)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  input/state  asynchronous  sequential  machine  with  adversarial  input, 
having  the  state  set  X  =  {x1,  ...,  xn}  and  the  complete  matrix  of  stable  transitions  9f(2,v).  The  control  skeleton 
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matrix  K(2,v)  of  2  is  an  nxn  matrix  of  zeros  and  ones,  whose  entries  are  defined  as  follows  for  each  i,  j  G  {1, 

2, n}:  Kij(2v)  :=  1  if  9^(2, v)  ^  N,  and  Kfj  :=  0  if  9ftij(2,v)  =  N.  ♦ 

(6.10)  EXAMPLE.  Using  the  result  of  Example  6.7,  we  obtain  the  control  skeleton  matrix 

/  1  0  0  \ 

K(2,v)  =  I  1  1  0  I  ♦ 

'101' 

In  these  terms,  a  state  feedback  controller  can  take  2  from  a  stable  combination  with  the  state  x1  to  a  stable 
combination  with  the  state  xJ  in  fundamental  mode  operation  if  and  only  if  Kfj(2,v)  =  1. 

6.1.  Latent  Adversarial  Switches. 

We  turn  our  attention  now  to  a  restricted  version  of  the  model  matching  problem  with  adversarial  inputs.  The 
solution  of  this  problem  forms  a  step  stone  along  our  way  toward  the  solution  of  the  full  model  matching  problem 
with  adversarial  inputs. 

Consider  an  asynchronous  machine  2  with  adversarial  input  uncertainty  v.  Note  that  changes  in  the 
adversarial  input  value  do  not  always  cause  a  state  transition  of  2.  Indeed,  assume  that  2  is  at  a  stable  combination 
with  the  state  x  and  the  control  input  character  u0,  and  consider  the  set  of  adversarial  input  characters  sa(x,u0,x)  of 
(5.1).  Clearly,  if  this  set  consists  of  more  than  one  character,  then  a  switch  of  the  adversarial  input  from  one 
character  to  another  in  this  set  does  not  change  the  stable  state  of  2,  and  hence  is  not  noticeable  by  a  state  feedback 
controller.  This  leads  to  the  following. 

(6.1 1)  DEFINITION.  A  latent  adversarial  switch  is  a  change  of  the  adversarial  input  value  that  does  not  result  in  a 
change  of  the  stable  state  of  the  machine.  ♦ 

The  next  statement  provides  a  solution  of  the  model  matching  problem  for  the  case  when  all  adversarial  input 
changes  are  latent. 

(6.12)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  input/state  machine  with  adversarial  input,  and  assume  that  the 
adversarial  input  is  restricted  to  latent  switches.  Let  K(2,v)  be  the  control  skeleton  matrix  of  2,  and  let  2'  = 
(A,X,s')  be  a  stable-state  input/state  machine  with  no  adversarial  input,  having  the  skeleton  matrix  K(2').  Then,  the 
following  two  statements  are  equivalent. 

(i)  There  exists  a  state  feedback  controller  C  for  which  2c|s  =  2',  where  the  closed  loop  machine  2c  is  well  posed 
and  operates  in  fundamental  mode. 

(ii)  K(2,v)  >  K(2'). 

(6.13)  EXAMPLE.  Consider  the  problem  of  building  a  model  matching  controller  for  the  machine  2  of  Example 
2.2  so  as  to  match  the  model  2'  of  Example  3.2.  Using  the  procedure  described  in  MURPHY,  GENG,  and 
HAMMER  [2002  and  2003],  the  skeleton  matrix  of  the  model  2'  is  calculated  as 

(l0°) 

K(2')  =1  10. 

M01' 

The  control  skeleton  matrix  K(2,v)  was  calculated  in  Example  6.10.  Comparing  the  two  matrices,  we  obtain  that 
K(2,v)  >  K(2'),  so  that  a  model  matching  controller  exists  by  Theorem  6.12.  We  will  construct  the  controller  in 
section  9  below.  ♦ 

Proof  (of  Theorem  6.12).  Assume  first  that  (i)  is  valid,  and  consider  a  particular  stable  transition  of  the  model  2', 
say  a  transition  from  a  stable  combination  with  the  state  x1  to  a  stable  combination  with  the  state  xJ.  Then,  by 
definition  of  the  skeleton  matrix,  K;j(2')  =  1.  As  (i)  is  valid,  the  stable  state  machine  2C|S  must  also  have  a  transition 
from  a  stable  combination  with  the  state  x1  to  a  stable  combination  with  xJ.  In  view  of  the  fact  that  2C  is  the 
machine  2  controlled  by  the  controller  C,  it  follows  by  Corollary  6.8  and  Definition  6.9  that  K;j(2,v)  =  1  as  well. 
Thus,  K;j(2,v)  =  1  if  K;j(2')  =  1,  i,  j  =  1,  2,  ...,  n.  Considering  that  K(2,v)  and  K(2')  are  both  matrices  of  zeros 
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and  ones,  the  latter  implies  that  K(2,v)  >  K(2'),  and  (i)  implies  (ii). 

Conversely,  assume  that  (ii)  is  valid.  Let  K'(2')  be  the  one-step  skeleton  matrix  of  the  model  2'  (see 
MURPHY,  GENG,  and  HAMMER  [2003]).  Then,  K(2')  >  K1(2'),  and  it  follows  by  (ii)  that 

(6.14)  K(2,v)  >  K‘(2'). 

Now,  let  i,  j  E  {1,2,  ...,  n}  be  a  pair  of  integers  for  which  Ky(2')  =  1.  Let  s'  be  the  stable  recursion  function  of  the 
model  2',  and  denote  by  V(i,j)  the  set  of  all  control  input  characters  vGA  for  which  s'(x‘,v)  =  xJ.  Note  that,  for 
all  i, 

(6.15)  V(i,j)nV(i,j')  =  0  if  j*j', 

since  different  target  states  require  different  inputs.  As  K*(2')  =  1,  it  follows  by  (6.14)  that  also  K;j(2,v)  =  1.  By 
Corollary  6.8,  there  is  then  a  complete  set  of  strings  from  the  state  x1  to  the  state  xJ.  Select  a  character  v  £  V(i,j). 

In  view  of  Theorem  5.7,  there  is  controller  F(x‘,xJ,v)  that  takes  the  machine  2  from  a  stable  combination  with  x1 
to  a  stable  combination  with  xJ;  here,  the  character  v  activates  the  controller.  Extend  the  activation  of  this 
controller  to  all  characters  v  £  V(i,j),  so  that  any  character  v  £  V(i,j)  can  be  used  to  start  the  (same)  controller 
action.  Denote  the  resulting  controller  by  F(x‘,xJ,V(i,j)). 

Next,  we  define  the  following  operation  of  join  for  combining  two  controllers  (see  also  VENKATRAMAN  and 
HAMMER  [2006c]).  Given  two  controllers  F(x‘,xj,V(i,j))  and  F(x'  ,xJ  ,V(i',j')),  ihejoin 

C  :=  F(xi,xj,V(i,j))  v  F(x‘,xJ  ,V(i' j')) 

is  constructed  as  follows: 

(i)  When  x1  =  x1  and  the  external  input  character  is  v,  then 

=  |F(x‘,xJ,V(i,j))  if  v  £  V(i,j), 

LF (x1  ,xJ  ,V (i',j '))  if  v£V(i',j'); 

(ii)  If  (x‘,xJ)  ^  (x‘,xJ ),  let  the  machine  2  be  in  a  stable  combination  with  the  state  x1,  when  the  controller  C 
receives  the  input  character  v.  Then, 

C  :=  F(xi,xj,V(i,j)). 

This  construction  of  C  is  consistent  by  (6.15)  (see  VENKATRAMAN  and  HAMMER  [2006c]  for  more  details). 

Now,  let  T  C  {1,  2,  ...,  n}  x  {1,  2,  ...,  n]  be  the  set  of  all  pairs  of  integers  for  which  K*  (2')  =  1.  Then,  a  slight 
reflection  shows  that  the  joined  controller 

F  :=  VijGTF(x‘,xJ,V(i,j)) 

makes  the  machine  2  match  the  model  2'.  Theorem  5.7,  the  closed  loop  machine  is  well  defined  and  operates  in 
fundamental  mode.  This  concludes  our  proof.  ♦ 


7.  GENERAL  MODEL  MATCHING 


7.1.  Adversarial  Detectability 


Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  with  adversarial  input.  Assume  that  2  is  at  a  stable 
combination  (x,u,w),  when  the  adversarial  input  value  changes  to  w'.  This  change  may  or  may  not  cause  2  to 
experience  a  transition  to  a  new  stable  state.  Presently,  consider  the  case  when  the  change  in  the  adversarial  input 
from  w  to  w'  causes  2  to  move  to  a  new  state  x'  ±  x.  We  refer  to  such  a  transition  as  an  adversarial  transition. 
In  this  section,  we  discuss  the  existence  and  the  design  of  a  state  feedback  controller  Ca  that  automatically 
counteracts  adversarial  transitions  of  2. 

A  basic  requirements  is,  of  course,  that  the  controller  Ca  operate  in  fundamental  mode  to  guaranty 
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deterministic  behavior  of  the  closed  loop  machine.  Being  a  state  feedback  controller,  Ca  has  access  to  the  current 
state  of  2,  and  it  generates  the  control  input  of  2.  To  obtain  fundamental  mode  operation  of  the  closed  loop 
machine  with  the  controller  Ca,  it  must  be  possible  to  determine  from  the  state  of  2  whether  or  not  2  has  reached 
its  next  stable  combination.  This  leads  us  to  the  following,  which  is  closely  analogous  to  Definition  4.4. 

(7.1)  DEFINITION.  Let  2  =  (AxB,X,f)  be  an  input/state  asynchronous  machine  with  adversarial  uncertainty  £. 
Assume  that  2  is  in  a  stable  combination  at  the  state  x  with  the  control  input  character  u,  when  a  change  in  the 
adversarial  input  causes  2  to  move  to  the  state  x'.  Then,  the  pair  (x,u)  is  adversarially  detectable  with  respect  to 
the  adversarial  uncertainty  £  if  it  can  be  determined  from  the  current  state  of  2  whether  or  not  2  has  reached  its 
next  stable  combination.  ♦ 

Without  adversarial  detectability,  it  is  not  possible  to  guaranty  fundamental  mode  operation  of  a  closed  loop 
machine  controlled  by  a  state  feedback  controller.  In  other  words,  operation  must  be  restricted  to  adversarially 
detectable  pairs  of  the  controlled  machine  2. 

Assume  then  that  the  machine  2  is  at  a  stable  combination  (x,u,w),  when  the  adversarial  input  character 
changes  to  w',  causing  2  to  transition  to  a  stable  combination  with  the  state  x'  ^  x.  This  transition  may,  of  course, 
consist  of  a  number  of  intermediate  steps,  say  x0  :=  x,  X[  :=  f(x0,u,w'),  x2  =  f(xbu,w'),  ...,  xq  :=  f(xq_!,u,w')  =  x',  xq  := 
f(xq,u,w').  Similarly  to  (4.1)  and  (4.2),  we  denote 

|0(x,u,w')  :=X!...xq, 

1 ’  L8[x,u,e]  :=  {0(x,u,w') :  w'  <E  e}. 

The  following  statement  is  closely  analogous  to  Theorem  4.5  and  has  a  similar  proof. 

(7.3)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  input/state  asynchronous  machine  with  adversarial  input.  Assume 
that  2  is  in  a  stable  combination  with  the  state  x  and  the  control  input  value  u.  In  the  notation  of  (3.3)  and  (7.2), 
the  following  two  statements  are  equivalent. 

(i)  The  pair  (x,u)  is  adversarially  detectable  with  respect  to  the  adversarial  uncertainty  v. 

(ii)  States  of  the  set  sv(x,u)  appear  only  at  the  end  of  strings  belonging  to  0[x,u,v].  ♦ 

As  indicated  earlier,  to  guaranty  fundamental  mode  operation  of  the  closed  loop  machine,  the  use  of  the 
machine  2  must  be  restricted  to  adversarially  detectable  pairs.  This  leads  us  to  the  following  notion.  (For  a  string  a 
=  w|uiU2...uq  €E  BxA1 ,  denote  by  a  :=  uq  the  last  control  input  character  of  the  string.) 

(7.4)  DEFINITION.  Let  2  =  (AxB,X,f,v)  be  an  asynchronous  machine  having  adversarial  uncertainty  v,  n  states, 
and  the  combined  matrix  of  stable  transitions  R(2,v).  The  reduced  matrix  of  stable  transitions  Rr(2,v)  of  2  is 
obtained  by  removing  from  each  column  j  =  1,  2, ...,  n  of  R(2,v)  all  strings  a  for  which  the  pair  (x],IIc  a)  is  not 
adversarially  detectable  with  respect  to  the  uncertainty  v.  ♦ 

(7.5)  EXAMPLE.  We  calculate  now  the  reduced  matrix  of  stable  transitions  for  the  matrix  R(2,v)  of  Example  6.4. 
Considering  the  transition  table  of  the  machine  2  as  provided  in  Example  2.2,  note  that  there  is  only  one  transition 
that  can  be  caused  by  the  adversarial  input,  namely,  the  transition  initiated  by  a  switch  of  the  adversarial  input  from 
the  character  a  to  the  character  (5,  while  2  is  at  the  state  x1  and  the  control  input  is  b.  Symbolically,  the 
transition  can  be  represented  by  (x',b,a)  -»  (x',b,P)  -*  (x2,b,P).  For  this  transition,  recalling  that  s  is  the  stable 
recursion  function  of  2,  we  have  s(x‘,b,P)  =  x2.  Hence,  0[x',b,a]  =  x1  and  0[x',b,p]  =  x2,  so  that 

0[x\b,v]  =  {x‘,x2}. 

In  this  case,  we  have 

sv(x1,b)={x1,x2}. 

As  we  can  see,  the  states  x1,  x2  appear  only  at  the  end  of  strings  belonging  to  0[x*,b,v],  Therefore,  by  Theorem  7.3, 
the  pair  (x',b)  is  adversarially  detectable,  and  we  have  Rr(2,v)  =  R(2,v)  in  this  case.  ♦ 

The  reduced  matrix  of  stable  transitions  characterizes  all  transitions  of  the  machine  2  that  end  at  adversarially 
detectable  pairs.  This  matrix  forms  the  basis  for  designing  controllers  that  can  counteract  adversarial  transitions. 
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7.2. 


Reversing  Adversarial  Transitions. 


Assume  that  the  machine  2  is  at  an  adversarially  detectable  stable  combination  with  the  pair  (xs,u)  G  XxA, 
when  a  change  in  the  adversarial  input  causes  2  to  move  to  a  stable  combination  with  the  state  x';  of  course,  by 
fundamental  mode  operation,  the  control  input  value  u  is  kept  constant  during  this  process.  As  this  transition  started 
from  an  adversarially  detectable  pair,  a  state  feedback  controller  can  determine  from  the  current  state  of  2  whether 
or  not  2  has  reached  its  next  stable  combination.  Having  been  caused  by  the  adversarial  input,  this  transition  is 
undesirable;  our  objective  is  to  design  a  state  feedback  controller  Ca  that  automatically  reverses  this  transition.  In 
this  way,  Ca  will  counteract  the  effects  of  the  adversarial  input. 

Consider  then  an  adversarial  transition  from  a  stable  combination  with  the  pair  (xs,u)  to  a  stable  combination 
with  the  pair  (x',u).  Letting  v  be  the  adversarial  uncertainty,  the  set  of  adversarial  input  characters  that  can  give 
rise  to  such  a  transition  is 

(7.6)  v(xs,x’,u)  :=  sa(xs,u,x*)  fj  v. 


Clearly,  this  transition  is  possible  if  and  only  if  v(xs,xl,u)  f  0,  and  we  reach  the  following. 

(7.7)  DEFINITION.  Let  2  be  an  asynchronous  machine  with  the  state  set  X  =  {x1,  x2, ...,  x"}  and  the  adversarial 
uncertainty  v,  and  assume  that  2  is  in  a  stable  combination  with  the  control  input  character  u.  Then,  for  a  pair  of 
integers  s,  t  G  (1,  2,  ...,  n},  the  adversarial  transition  indicator  is 


(7.8) 


K(xs,x',u) 


1  if  v(xs,x*,u)  f  0,  ^ 

0  otherwise. 


The  discussion  leading  to  Definition  7.7  implies  the  next  statement. 

(7.9)  LEMMA.  Assume  that  the  closed  loop  machine  2  is  in  a  stable  combination  at  the  state  xs  with  the  control 
input  character  u.  Then,  the  following  two  statements  are  equivalent. 

(i)  There  is  an  adversarial  transition  to  a  stable  combination  with  the  state  x*. 

(ii)  K(xs,x',u)  =  1.  ♦ 

To  address  the  question  of  whether  an  adversarial  transition  is  reversible  or  not,  it  is  convenient  to  introduce  the 
following. 

(7.10)  DEFINITION.  Let  2  be  an  asynchronous  machine  with  adversarial  input.  An  adversarial  transition  from  a 
stable  combination  with  the  state  xs  to  a  stable  combination  with  the  state  x1  is  reversible  if  there  is  a  state 
feedback  controller  that  drives  2  back  to  a  stable  combination  with  the  state  xs,  without  specific  information  about 
the  adversarial  character  that  caused  the  transition.  ♦ 


To  examine  reversible  transitions,  assume  that  the  machine  2  is  in  a  stable  combination  with  the  state  xs  and 
the  control  input  character  u,  when  an  adversarial  transition  to  the  state  x*  occurs.  In  view  of  (7.6),  the  adversarial 
uncertainty  immediately  after  the  transition  is  v(xs,x‘,u).  Using  the  reduced  matrix  of  stable  transitions  Rr(2,v),  we 
construct  the  scalar  function 


(7.11) 


Kr(xs,x\u) 


1  if  R[s(2,v)  includes  a  complete  set  of  strings 

with  respect  to  the  adversarial  uncertainty  v(xs,x',u), 
-0  otherwise. 


A  slight  reflection  indicates  that  the  following  is  true. 

(7.12)  LEMMA.  In  the  above  notation,  an  adversarial  transition  from  xs  to  x‘  is  reversible  if  and  only  if  Kr(xs,x',u) 
=  1.  ♦ 

Next,  let  U(xs)  C  A  be  the  set  of  all  control  input  characters  of  the  machine  2  that  may  appear  in  stable 
combinations  with  the  state  xs.  For  various  practical  considerations,  the  designer  of  the  state  feedback  controller  C 
may  choose  to  avoid  using  some  of  these  control  input  characters.  The  set  of  active  control  input  characters  of  2  at 
the  state  xs  is  the  subset  S(xs)  C  U(xs)  of  all  control  input  characters  that  may  be  applied  by  the  controller  C, 
while  the  closed  loop  machine  2C  is  in  a  stable  combination  at  the  state  xs  of  2.  If  xs  does  not  appear  as  part  of  a 
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stable  state  of  the  closed  loop  machine,  then  S(xs)  :=  0.  When  all  states  of  2  appear  as  stable  states  of  the  closed 
loop  machine  2C  and  all  possible  control  input  characters  of  2  are  utilized,  we  have 

(7.13)  S(xs)=U(xs),  s=  1,2,  ...,  n. 

Equality  (7.13)  describes  the  most  common  situation.  When  it  is  valid,  we  say  that  there  are  no  idle  control  input 
characters. 

(7.14)  DEFINITION.  Let  2  be  an  asynchronous  machine  with  adversarial  input  having  the  n  states  x1, ...,  x".  Let 
S(xs)  be  the  set  of  active  control  input  characters  at  the  state  xs.  The  reversal  matrix  A(S,2)  is  an  nxn  numerical 
matrix  with  the  entries 

a  /q  y1*  _ imin  {kF(xV,u)  “  K(xs,x',u)  :  u  G  S(xs)}  if  S(xs)  +  0, 

Ast(i>,2)  .  ^  .f  = 

s,  t  =  1,  2,  ...,  n.  ♦ 

For  a  numerical  matrix  D,  the  inequality  D  >  0  means  that  D  has  no  negative  entries. 

(7.15)  EXAMPLE.  We  continue  our  examination  of  the  machine  2  of  Example  2.2.  In  view  of  Example  7.5,  there 
is  only  one  adversarial  transition  we  have  to  consider,  namely,  the  transition  x2  =  s(x',b,P).  Assume  that  the 
controller  is  implemented  with  no  idle  control  input  characters,  so  that  S(x')  =  U(x').  From  (7.6),  we  have 

v(x1,x2,b)  =  sa(x1,b,x2)nv  =  { (3}  D{  ot,(3}  =  {p}. 

From  (7.8),  we  get  that  K(x',x2,b)  =  1.  In  Example  7.5,  we  have  seen  that  Rr(2,v)  =  R(2,v).  Also,  from  Example 
6.7,  we  have  that  the  complete  set  of  strings  D^i^v)  =  {o.|ab,(3|a,(3|ba} .  Consequently,  R2i(2,v)  includes  a 
complete  set  of  strings  with  respect  to  the  adversarial  uncertainty  v(x',x2,b)  =  { |3} .  Substituting  into  (7.1 1),  we 
obtain  that  Kr(x',x2,b)  =  1,  so  that  Aj2(U,2)  =  0.  As  there  are  no  adversarial  transitions  other  than  the  transition 
from  x1  to  x2,  we  have  that  K(xs,x',u)  =  0  for  all  (s,t)  ^  (1,2).  Whence,  A(U, 2)  >  0  in  this  case.  ♦ 

We  can  characterize  now  the  conditions  under  which  a  transition  caused  by  the  adversarial  input  can  be 
counteracted  by  a  state  feedback  controller. 

(7.16)  THEOREM.  Let  2  be  an  asynchronous  machine  with  adversarial  input,  and  let  X  =  {x1, ...,  x11}  be  the  state 
set  of  2.  Assume  that  2  is  operated  by  a  state  feedback  controller  using  the  active  control  input  character  sets  S(xs), 
s  =  1,  2,  ...,  n,  and  let  A(S,2)  be  the  corresponding  reversal  matrix.  Then,  the  following  two  statements  are 
equivalent. 

(i)  All  adversarial  transitions  of  the  closed  loop  machine  can  be  automatically  reversed  in  fundamental  mode 
operation. 

(ii)  A(S,2)  >  0. 

Proof.  Consider  an  adversarial  transition  from  a  stable  combination  with  the  state  xs  to  a  stable  combination  with 
the  state  x'.  We  have  two  cases  here,  depending  on  the  set  S(xs)  of  active  control  input  characters: 

Case  1:  S(xs)  =  0:  then,  the  state  xs  does  not  appear  in  a  stable  combination  of  the  closed  loop  machine,  and  hence 
no  adversarial  transitions  can  start  at  the  state  xs.  Then,  Ast(S,2)  =  0  by  Definition  7.14. 

Case  2:  S(xs)  ^  0:  By  Lemma  7.9,  an  adversarial  transition  from  xs  to  xl  is  possible  if  and  only  if  K(xs,x',u)  =  1 
for  a  control  input  value  u  G  S(xs).  By  Lemma  7.12,  this  transition  can  be  reversed  if  and  only  if  Kr(xs,x',u)  =  1.  As 
K(xs,x',u)  and  K'(xs,xl,u)  can  only  take  the  values  0  or  1,  we  conclude  that  the  adversarial  transition  from  xs  to 
x*  is  reversible  if  and  only  if  Kr(xs,xl,u)  -  K(xs,x',u)  >  0.  As  this  is  true  for  all  states  xs  and  x'  and  for  all  input 
values  u  G  S(xs),  it  follows  that  (i)  and  (ii)  are  equivalent.  This  concludes  our  proof.  ♦ 

Most  often,  a  state  feedback  controller  employs  every  control  input  character  of  the  set  U(xs).  In  such  case,  the 
combination  of  Theorems  (6.12)  and  (7.16)  yields  the  following  statement,  which  is  the  main  result  of  this  section. 

(7.17)  THEOREM.  Let  2  =  (AxB,X,f,v)  be  an  input/state  machine  with  adversarial  input,  and  assume  that  2  has 
no  idle  control  input  characters.  Let  U(x)  be  the  set  of  control  input  characters  that  appear  in  stable  combinations 
with  the  state  x,  let  A(U,2)  be  the  corresponding  reversal  matrix,  and  let  K(2,v)  be  the  control  skeleton  matrix  of 
2.  Let  2'  =  (A,X,s')  be  a  stable-state  input/state  machine  with  no  adversarial  input,  having  the  skeleton  matrix 
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K(2').  Then,  the  following  two  statements  are  equivalent: 

(i)  There  is  a  controller  C  for  which  2c|s  =  2',  where  2c  is  well  posed  and  operates  in  fundamental  mode. 

(ii)  K(2,v)  >  K(2')  and  A(U,2)>0.  ♦ 

Theorem  7.17  provides  a  comprehensive  solution  to  the  model  matching  problem  for  asynchronous  machines 
with  adversarial  inputs.  As  we  can  see,  the  solution  is  entirely  characterized  by  two  numerical  matrix  inequalities. 
As  the  model  machine  2'  has  no  adversarial  input,  the  controller  C,  when  it  exists,  counteracts  any  effects  of 
adversarial  activity. 

(7.18)  EXAMPLE.  Combining  now  the  results  of  Examples  6.13  and  7.15,  we  conclude  that  condition  (ii)  of 
Theorem  7.17  is  valid  for  the  machine  2  of  Example  2.2  and  the  model  2'  of  Example  3.2.  Therefore,  Theorem 
7.17  assures  us  that  there  is  a  controller  C  that  controls  2  so  that  the  closed  loop  machine  2C  is  stably  equivalent 
to  the  model  2',  thus  solving  the  perturbed  model  matching  problem  in  this  case.  The  construction  of  the  controller 
C  is  described  in  section  9  below.  ♦ 


8.  CONTROLLER  STRUCTURE 


We  summarize  now  the  structure  of  a  controller  that  solves  the  model  matching  problem  with  adversarial  inputs. 
In  general  terms,  the  controller  consists  of  two  components:  a  component  that  achieves  model  matching  and  a 
component  that  reverses  adversarial  transitions. 

Consider  the  problem  of  controlling  the  machine  2  to  match  the  model  2'.  The  construction  of  the  model 
matching  component  of  the  controller  is  described  in  the  proof  of  Theorem  6.12.  Regarding  adversarial  transitions  - 
these,  by  their  nature,  occur  while  the  model  2'  remains  in  a  stable  combination,  since  the  model  has  no  adversarial 
input.  Therefore,  an  adversarial  transition  can  be  characterized  as  follows  (refer  to  the  diagram  below):  it  is  a 
departure  from  a  stable  combination  of  the  closed  loop  2C  that  is  not  preceded  by  a  change  of  the  external  command 
input  v.  We  describe  next  the  controller  in  brief  terms. 


(i)  The  state  feedback  controller  F  controls  the  machine  2  to  achieve  model  matching  in  response  to  the  external 
command  input  v.  The  controller  F  is  built  following  the  procedure  described  in  the  proof  of  Theorem  6.12. 

(ii)  The  comparator  detects  stable  states  of  2C  that  differ  from  stable  states  of  2'.  When  such  a  difference  is 
detected,  the  comparator  activates  the  state  feedback  controller  F  to  drive  2  back  to  a  stable  combination  with  the 
correct  state. 

An  example  of  the  construction  of  the  controller  C  is  provided  in  the  next  section. 
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9.  EXAMPLE 


In  this  section,  we  construct  a  controller  C  that  solves  the  model  matching  problem  for  the  machine  2  of 
Example  2.2  and  the  model  2'  of  Example  3.2;  the  adversarial  uncertainty  is  v  =  {a,  (3} .  It  can  be  shown  that 
condition  (ii)  of  Theorem  7.17  is  satisfied  in  this  case.  Below,  we  demonstrate  the  construction  of  a  model  matching 
controller.  Recall  that  s  is  the  stable  transition  function  of  2  and  s'  is  the  stable  transition  function  of  2'.  An 
examination  of  the  transition  tables  of  the  machines  2  and  2'  shows  that  only  the  following  three  transitions  of  2 
are  different  from  corresponding  transitions  of  2': 

s(x1,b,(3)  =  x2  ::  s'(x1,b)  =  x1; 
s(x2,a,a)  =  x3  ::  s'(x2,a)  =  x  , 
s(x3,b,P)  =  x2  ::  s'(x3,b)  =  x'. 

Using  the  procedure  described  in  the  proof  of  Theorem  6.12,  we  build  three  state  feedback  controllers 
F(x1,x1,b),  F(x2,x',a),  and  F(x3,x',b),  each  of  which  respectively  "corrects"  one  of  these  transitions. 

1)  Construction  of  the  controller  F(x1,x1,b): 

We  use  the  state  set  {^°,  S.2,  ?3}  for  FCx^x^b).  Note  that  (x*,a)  is  a  detectable  pair.  Upon  detecting  a 

detectable  transition  of  2  to  the  state  x1  with  the  control  input  value  a,  the  controller  F(x1,x1,b)  moves  to  a  stable 
combination  with  its  state  while  continuing  to  apply  to  2  the  input  character  it  receives: 

(K§°,(z,t))  :=  for  all  (z,t)  +  (x‘,a), 

<Ki°,(x\a))  :=  i1, 

<K?\(x\a))  :=  i‘. 

il(i°,(z,t))  :=t  for  all  (z,t)EXxA, 
rid'^zd))  :=  t  for  all  (z,t)  £  XxA, 

Next,  upon  receiving  the  external  input  character  b,  the  controller  F(x1,x1,b)  moves  to  a  stable  combination  with  its 
state  §2,  namely: 

<Ki\(z,t))  :=  for  all  (z,t)  +  (x‘,b), 

•KiV.b))  :=  |2, 

•KiV.b))  :=  |2. 

At  this  point,  the  controller  F(x1,x1,b)  must  start  to  generate  a  string  of  control  input  characters  to  keep  2  at  the 
state  x1.  An  examination  of  the  entry  9^  i  i(2,v)  in  Example  6.7  shows  that  the  single  control  input  character  a 
satisfies  this  requirement.  So  we  set  the  output  function  of  the  controller  F(x1,x1,b)  as 

r)(§2,(x1,t))  :=  a  for  all  t  £  A. 

When  the  controller  F(x1,x1,b)  detects  the  state  x1  of  2,  it  moves  to  a  stable  combination  with  the  state  |3,  namely, 

c|>(i2,(z,t))  :=  |2  for  all  (z,t)  +  (x‘,a), 

<K?2,(x‘,b))  :=  |3, 

^.(x'.b))  :=  |3, 

and  continues  to  generate  the  control  input  character  a: 
ri(§3,(xI,t))  :=  a  for  all  t£A. 

Finally,  upon  a  change  of  the  external  input  character,  F(x1,x1,b)  resets  to  its  initial  state  S=°: 

<Ki3.(z»t))  :=  for  all  (z,t)  +  (x‘,b). 

2)  The  constmction  of  the  controllers  F(x2,x‘,a)  and  F(x3,x',b)  is  similar  to  the  construction  of  F(x1pc1,b). 

3)  Counteracting  adversarial  transitions: 

Recall  from  Example  7.15  that  the  machine  2  has  only  one  transition  that  can  be  caused  by  the  adversarial 
input  -  the  transition  x2  =  s(x 1  ,b,(J< )  when  the  adversarial  input  character  changes  from  a  to  (3.  Assume  then  that 
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the  machine  2  is  at  a  stable  combination  with  the  pair  (x',b),  when  the  state  of  2  switches  to  x2.  By  Example 
7.15,  the  adversarial  uncertainty  is  then  v(x',x2,b)  =  {|3}.  We  build  now  a  controller  Fa(x  ,x')  that  counteracts  this 
action  of  the  adversarial  input,  and  returns  2  to  the  state  x1  immediately  after  the  comparator  detects  the  transition 
to  x2.  The  output  d  of  the  comparator  can  take  two  values:  d  :=  1  when  the  stable  state  of  2C  is  different  from  the 
stable  state  of  2',  and  d  :=  0  when  the  two  stable  states  are  equal. 

Referring  to  Examples  6.7,  7.5,  and  7.15,  we  have  that  the  complete  set  of  strings  {a|ab,(3|a,(3|ba}  is  included 
in  the  entry  9t2i(2,v);  that  Rr(2,v)  =  R(2,v);  and  that  v(x1,x2,b)={(l}.  Consequently,  the  single  character  a  forms  a 
control  input  string  that  takes  2  back  to  the  state  x1.  The  construction  of  the  controller  Fa(x2,x')  is  reminiscent  of 
the  construction  of  the  controller  F(x1,x1,b)  we  have  described  earlier,  and  is  as  follows.  Let  cp  be  the  recursion 
function  of  Fa(x2,x*),  let  p  be  its  output  function,  and  let  {£°,  'Q  ,  g2}  be  its  state  set.  Starting  at  a  stable 
combination  with  the  pair  (x2,b),  set 

cp(C°,(z,t,0))  :=  C°  for  all  (z,t)  +  (x2,b); 
p(^°,(z,t))  :=  t  for  all  (z,t)  £  XxA; 
cp(C°,(x2,b,l))  :=  C1; 
q)(C‘,(x2,b,d))  :=  ^,dG{0,  1}; 
u('c',(z,t))  :=  a  for  all  (z,t)£XxA; 

cp(CI,(xI,b,d))  :=C2,d£{0,  1}; 

«p(£V,b,d))  :=?2,d£{0,  1}; 

p(t,2,(z,t))  :=  a  for  all  (z,t)  £  XxA; 
cp(C2,(z,t,d))  :=  for  all  (z,t)  +  (x‘,b). 

Finally,  we  assemble  the  combined  state  feedback  controller  F  by  using  the  join  operation  employed  in  the 
proof  of  Theorem  6.12: 

F  =  F(xV,b)  v  F(x2,x‘,a)  v  F(x3,x‘,b)  v  Fa(x2,x'). 

When  this  controller  is  inserted  into  the  control  diagram  (8.1),  it  eliminates  the  effects  of  the  adversarial  input  and 
makes  2  behave  like  the  deterministic  and  unperturbed  model  2'  of  Example  3.2. 
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